-- Logs begin at Thu 2022-06-30 15:19:05 UTC, end at Thu 2022-06-30 16:51:02 UTC. -- Jun 30 16:12:03.332262 dv-cinder-836027-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 30 16:12:03.344721 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 30 16:12:03.347776 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 30 16:12:03 2022] *** Jun 30 16:12:03.347776 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 30 16:12:03.347776 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 30 16:12:03.347776 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: nodename: dv-cinder-836027-2-iscsi Jun 30 16:12:03.347776 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: machine: x86_64 Jun 30 16:12:03.347776 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: clock source: unix Jun 30 16:12:03.347776 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: pcre jit disabled Jun 30 16:12:03.347776 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: detected number of CPU cores: 4 Jun 30 16:12:03.348245 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: current working directory: / Jun 30 16:12:03.348245 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: detected binary path: /usr/bin/uwsgi-core Jun 30 16:12:03.348245 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: your processes number limit is 47680 Jun 30 16:12:03.348245 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: your memory page size is 4096 bytes Jun 30 16:12:03.348245 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: detected max file descriptor number: 2048 Jun 30 16:12:03.348245 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: lock engine: pthread robust mutexes Jun 30 16:12:03.348245 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: thunder lock: enabled Jun 30 16:12:03.348592 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 30 16:12:03.348592 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 30 16:12:03.395455 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: Python main interpreter initialized at 0x559ae11039c0 Jun 30 16:12:03.395455 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: python threads support enabled Jun 30 16:12:03.395455 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: your server socket listen backlog is limited to 100 connections Jun 30 16:12:03.395455 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: your mercy for graceful operations on workers is 90 seconds Jun 30 16:12:03.395884 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: mapped 403077 bytes (393 KB) for 2 cores Jun 30 16:12:03.395884 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: *** Operational MODE: preforking *** Jun 30 16:12:03.396184 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: *** uWSGI is running in multiple interpreter mode *** Jun 30 16:12:03.396184 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: spawned uWSGI master process (pid: 80285) Jun 30 16:12:03.396401 dv-cinder-836027-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 30 16:12:03.397123 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: spawned uWSGI worker 1 (pid: 80297, cores: 1) Jun 30 16:12:03.398857 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: spawned uWSGI worker 2 (pid: 80298, cores: 1) Jun 30 16:12:03.398857 dv-cinder-836027-2-iscsi devstack@keystone.service[80285]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 30 16:12:08.968938 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 16:12:08.968938 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: from cryptography.utils import int_from_bytes Jun 30 16:12:08.969927 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 16:12:08.969927 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: from cryptography.utils import int_from_bytes Jun 30 16:12:08.971175 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 16:12:08.971175 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: from cryptography.utils import int_from_bytes Jun 30 16:12:08.971726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 16:12:08.971726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: from cryptography.utils import int_from_bytes Jun 30 16:12:09.022424 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 30 16:12:09.022603 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 30 16:12:09.022713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 30 16:12:09.022810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 30 16:12:09.022892 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] command line args: [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 30 16:12:09.022974 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] command line args: [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 30 16:12:09.023053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 30 16:12:09.023139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 30 16:12:09.023219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ================================================================================ {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 30 16:12:09.023299 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ================================================================================ {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 30 16:12:09.023430 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] admin_token = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.023578 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] admin_token = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.023740 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] config_dir = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.023850 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] config_dir = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.023932 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.024012 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.024090 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] config_source = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.024172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] config_source = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.024250 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] debug = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.024331 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] debug = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.024410 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.024701 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.024799 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.024871 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.024936 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.025000 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.025665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.026382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.026382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.027289 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.027357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.027423 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.027654 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_config_append = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.027794 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_config_append = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.027911 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.027994 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.028176 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_dir = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.028266 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_dir = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.028333 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_file = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.028410 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_file = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.028579 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.028719 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.028822 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.028897 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.029015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.029109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.029225 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.029225 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.029417 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.029495 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.029620 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.029703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.029788 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.029892 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.029969 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.030089 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.030164 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.030293 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.030293 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.030487 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.030564 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.030686 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.030762 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.030929 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.031055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.031146 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.031225 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.031307 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.031307 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.031514 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.67/identity {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.031514 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] publish_errors = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.031710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.031789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.031789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.032068 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.67/identity {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.032155 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.032222 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] publish_errors = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.032303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.032382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.032459 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.032536 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.032633 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] standard_threads = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.032717 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.032796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.032873 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.032940 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.033016 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.033169 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.033249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] standard_threads = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.033326 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] use_journal = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.033475 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.033558 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] use_json = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.033558 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] use_stderr = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.033706 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.033785 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] use_syslog = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.033863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.033941 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.034018 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] use_journal = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.034018 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] use_json = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.034165 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.034243 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] use_stderr = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.034321 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.034393 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] use_syslog = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.034470 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.034548 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 16:12:09.034628 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.034701 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.034816 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.034911 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.035052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.035132 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.035296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.035382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.035552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.045874 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.046014 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.046100 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.046191 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.046267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.046344 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.046417 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.046494 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.046567 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.046646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.046719 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.046791 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.046870 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.046949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.047022 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.047102 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.047176 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.047253 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.047331 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.047409 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.047497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.047655 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.047732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.047811 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.047884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.047956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.048033 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.048119 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.048269 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.048533 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.048617 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.048689 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.048768 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.048853 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.048928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.049000 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.049000 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.049134 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.049215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.049294 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.049372 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.049445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.049521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.049594 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.049675 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.049764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.049839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.049914 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.049991 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.050064 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.050223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.050341 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.050447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.050545 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.050675 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.050786 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.050893 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.050991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.051087 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.051191 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.051289 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.051384 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.051576 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.051659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.051773 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.051773 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.051945 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.052025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.052109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.052184 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.052313 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.052431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.052507 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.052623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.052706 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.052789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.052863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.052981 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.053065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.053139 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.053256 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.053340 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.053415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.053538 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.053655 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.053732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.053848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.053937 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.054018 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.054097 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.054171 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.054287 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.054364 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.054445 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.054525 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.054594 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.054664 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.054730 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.054800 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.054878 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.054951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.055018 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.055089 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.055167 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.055248 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.055248 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.055386 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.055457 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.055544 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.055633 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.055708 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.055792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.055866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.055933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.056004 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.056078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.056149 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.056235 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.056309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.056458 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.056534 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.056534 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.056731 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.056813 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.056888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.056971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.057055 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.057129 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.057201 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.057284 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.057366 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.057444 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.057523 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.057602 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.057669 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.058445 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.058531 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.058604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.058667 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.058727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.058786 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.058845 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.058911 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.058981 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.059050 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.059122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.059200 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.059286 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.059366 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.059449 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.059555 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.059630 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.059708 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.059775 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.059847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.059922 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.059996 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.060063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.060135 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.060209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.060287 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.060362 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.060436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.060503 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.060574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.060641 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.060726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.060807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.060869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.060928 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.060987 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.061046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.061046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.061171 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.061245 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.061306 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.061365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.061424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.061504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.061567 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.061626 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.061685 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.061685 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.061794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.061862 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.061923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.061987 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.062060 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.062127 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.062199 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.062199 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.external = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.062331 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.062392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.062461 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.062527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.062593 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.062667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.062740 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.062837 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.password = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.062915 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.062976 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] auth.token = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.063067 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.063137 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.063199 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.external = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.063266 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.063328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.063387 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.063446 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.063537 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.063606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.063676 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.063737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.password = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.063797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.063856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] auth.token = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.063915 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.063987 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.064056 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.064118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.064183 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.064249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.064309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.064375 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.064444 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.064521 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.064582 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.064640 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.064699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.064758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.064824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.064898 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.064965 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.065026 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.065100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.065100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.065289 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.065365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.065425 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.065485 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.065544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.065760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.065833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.065896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.065956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.066024 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.066090 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.066155 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.066224 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.066298 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.066359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.066418 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.066477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.066535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.066610 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.066704 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.066769 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.066849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.066912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.066976 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.067045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.067122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.067184 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.067243 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.067303 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.067363 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.067434 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.067572 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.067650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.067770 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.067894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.067996 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.068109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.068211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.068309 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.068390 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.068548 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.068640 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.068779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.068860 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.069012 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.069095 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.069256 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.069351 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.069484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.069568 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.069717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.069799 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.069951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.070058 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.070182 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.070260 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.070413 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.070502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.070641 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.070730 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.070868 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.070955 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.071107 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.071188 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.071339 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.071427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.071610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.071727 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.071853 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.071934 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.072279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.072825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.072954 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.073076 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.073206 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.073332 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.074139 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.074416 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.074613 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.074695 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.074769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.074842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.075025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.075145 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.075145 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.075286 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.075445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.075445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.075599 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.085824 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.085919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.085991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.086064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.086137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.086208 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.086348 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.086430 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.086503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.086575 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.086646 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.086717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.086846 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.086922 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.087000 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.087073 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.087144 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.087217 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.087296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.087369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.087447 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.087539 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.087600 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.087659 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.087730 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.087792 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.087851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.087909 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.087968 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.088026 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.088098 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.088160 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.088218 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.088277 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.088344 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.088404 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.088471 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.088537 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.088537 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.088643 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.088701 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.088766 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.088766 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.088886 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.088886 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.088989 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.089047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.089116 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.089392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.089900 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.089972 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.090033 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.090092 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.090174 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.090236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.090302 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.090369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.090430 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.090490 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.090557 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.090618 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.090677 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.090736 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.090794 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.090852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.090919 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.090980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.091039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.091105 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.091165 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.091224 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.091290 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.091351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.091410 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.091497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.091562 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.091622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.091693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.091755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.091813 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.091872 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.091929 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.091987 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.092053 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.092114 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.092191 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.092250 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.092308 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.092373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.092440 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.092500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.092559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.092617 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.092675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.092733 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.092733 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.092846 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.092904 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.092964 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.093028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.093088 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.093225 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.093289 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.093347 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.093406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.093464 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.093524 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.093591 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.093651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.093709 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.093768 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.093826 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.093884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.093949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.094010 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.094068 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.094127 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.094197 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.094257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.094326 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.094387 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.094445 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.094504 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.094561 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.094620 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.094620 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.094733 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.094791 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.094849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.094917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.094977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.095044 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.095104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.095163 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.095221 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.095279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.095338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.095403 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.095462 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.095542 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.095602 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.095677 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.095739 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.095806 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.095866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] role.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.095924 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.095983 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] role.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.096041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.096100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.096184 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.096245 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.096304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.096363 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.096422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.096488 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.096549 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.096607 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.096665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.096723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.096782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] role.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.096848 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.096914 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] role.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.096974 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.097033 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.097091 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.097150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.097215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.097276 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.097335 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.097394 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.097453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.097512 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.097579 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.097639 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.097698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.097757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.097757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.097862 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.097862 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.097975 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.098034 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.098104 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.098104 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.098214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.098281 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.098342 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.098401 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.098459 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.098518 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.098576 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.098576 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.098690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.098748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.098816 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.098875 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.098934 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.099003 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.099064 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.099122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.099181 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.099239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.099304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.099365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.099424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.100017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.101509 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.101613 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.101722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.101821 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.101901 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.102018 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.102018 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.102152 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.102263 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.102385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.102462 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.102523 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.102582 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.102647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.102772 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.102856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.102929 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.103001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.103062 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.103183 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.103275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.103351 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.103351 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.103495 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.103570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.103642 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.103725 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.103800 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.103800 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.103976 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.104052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.104123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.104247 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.104324 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.104396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.104467 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.104569 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.104666 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.104752 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.105345 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.105437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.105500 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.105560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.105620 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.105689 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.connection = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.105760 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.105828 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.105889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.105948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.106007 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.106007 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.106119 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.106190 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.106315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.106393 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.106465 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.106537 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.106621 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.106621 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.106752 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.106823 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.106895 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.connection = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.107063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.107148 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.107221 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.107301 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.107374 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.107445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.107580 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.107658 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.107730 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.107815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.107889 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.107961 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.108032 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.108104 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.108175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.108255 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.108399 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.108481 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.108630 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.108711 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.108783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80298) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 30 16:12:09.108944 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.109182 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.109482 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.109724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.109973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.110205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.110447 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.110685 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 16:12:09.110861 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80297) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 30 16:12:09.140805 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.141004 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.141233 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.142286 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.142516 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.142747 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.149790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.151925 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.179989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.180198 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.180688 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.180925 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.327583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.343550 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.393154 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.393412 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.393612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.393851 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.481893 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.482091 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.482203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.482312 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.482404 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.482562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.482645 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.482721 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.482844 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.482924 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.483008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.483236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.525076 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.525270 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.525346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.525483 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.525590 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.525719 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.525800 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.525914 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.525997 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.526129 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.526209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.526330 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.526409 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.526767 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.538060 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.538325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.551797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.552027 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.552254 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.552362 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.552477 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.552563 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.552695 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.552861 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.552951 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.553041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.553122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80298) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.553198 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.553362 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.553575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80297) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 16:12:09.571295 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.571508 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 16:12:09.571850 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.571939 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.572050 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 16:12:09.572133 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 16:12:09.572318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.572442 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.572580 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.572664 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 16:12:09.572838 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.572930 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.573094 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.573181 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.573455 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.573725 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.584123 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.584347 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.584614 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.584867 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.585001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.585241 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.585513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.585794 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.593996 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.594733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.597312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.597589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.597873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.598048 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.598177 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.598329 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.598455 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.598617 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.598744 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.598899 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.599038 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.599197 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.599423 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.599720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.599883 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.600043 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.600240 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.600412 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.600730 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.600965 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.600965 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.600965 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.627031 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.628215 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.628690 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.628976 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.629841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.630123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.634239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.634514 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.634913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.635408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.635705 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.636174 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.643571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.643853 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.644199 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.644483 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.648509 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.648794 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.648952 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.649086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.649249 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.649402 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.649547 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.649679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.649807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.649934 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.650062 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.650279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.650447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.650585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.650726 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.650855 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.670217 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.670493 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.670869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.671154 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.675286 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.675743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.679069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.679362 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.679465 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.679690 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.684836 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.685013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.688481 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.688658 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.692170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.692370 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.692522 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.692659 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.692917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.693070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.693589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.693779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.693915 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.694001 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.694001 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.694245 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.712614 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.712919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.713164 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.713297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.713441 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.713569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.713776 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.715212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.721870 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.722822 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.723589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.723866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.724556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.724833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.727081 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.727363 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.727665 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.727945 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.728151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.728307 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.728441 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.728700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.728976 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.729260 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.741814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.742105 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.742376 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.742658 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.742931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.743066 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.743234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.743530 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.743812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.744094 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.744769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.745079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.746680 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.747031 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.764770 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.766686 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.768200 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.770127 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.771616 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.773429 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.773662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.775057 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.775338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.775497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.775630 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.777970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.778241 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.778532 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.783990 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.785660 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.786239 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.788203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.789029 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.789340 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.789756 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.790036 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.791713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.792017 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.792978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.793363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.800437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.800739 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.801021 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.802949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.803254 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.803568 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.809327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80298) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.809622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.809906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.810183 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.810457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.810855 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.811138 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.811420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.811731 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.812010 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.812146 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80297) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 16:12:09.812278 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.812403 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.812540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.812667 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.812792 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.812917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.813046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80298) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.813173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.813507 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.813796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.814069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.814360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.814630 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.814907 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.815170 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.815434 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.815729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80297) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 16:12:09.851973 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559ae11039c0 pid: 80298 (default app) Jun 30 16:12:09.855817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559ae11039c0 pid: 80297 (default app) Jun 30 16:12:09.856269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fe1fb-43cf-4894-a0f4-7b3d961d5691 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:09.856576 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fe1fb-43cf-4894-a0f4-7b3d961d5691 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:09.856855 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31fe1fb-43cf-4894-a0f4-7b3d961d5691 None None] PATH_INFO: `/v3` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:09.861182 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1/1] 10.222.0.67 () {50 vars in 791 bytes} [Thu Jun 30 16:12:09 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:18.905341 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3478c56-2f8c-4549-9109-3d2d55eb214a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:18.905928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3478c56-2f8c-4549-9109-3d2d55eb214a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:18.905928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3478c56-2f8c-4549-9109-3d2d55eb214a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:18.910283 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1/2] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:12:18.915810 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397817fb-ec84-4462-84c6-245bf31f403d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:18.916207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397817fb-ec84-4462-84c6-245bf31f403d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:18.916367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397817fb-ec84-4462-84c6-245bf31f403d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:18.986373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: INFO dbcounter [None req-397817fb-ec84-4462-84c6-245bf31f403d None None] Registered counter for database keystone Jun 30 16:12:19.009381 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG oslo_db.sqlalchemy.engines [None req-397817fb-ec84-4462-84c6-245bf31f403d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80298) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 30 16:12:19.016295 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writer thread running {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 30 16:12:19.196279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG passlib.handlers.bcrypt [None req-397817fb-ec84-4462-84c6-245bf31f403d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80298) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 30 16:12:19.196599 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG passlib.handlers.bcrypt [None req-397817fb-ec84-4462-84c6-245bf31f403d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80298) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 30 16:12:19.251870 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-397817fb-ec84-4462-84c6-245bf31f403d None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:19.253496 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-397817fb-ec84-4462-84c6-245bf31f403d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:19.377061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 2/3] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 30 16:12:19.574585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3478c56-2f8c-4549-9109-3d2d55eb214a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:19.645832 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: INFO dbcounter [None req-a3478c56-2f8c-4549-9109-3d2d55eb214a None None] Registered counter for database keystone Jun 30 16:12:19.666492 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG oslo_db.sqlalchemy.engines [None req-a3478c56-2f8c-4549-9109-3d2d55eb214a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80297) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 30 16:12:19.673575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writer thread running {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 30 16:12:19.802708 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3478c56-2f8c-4549-9109-3d2d55eb214a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:19.807605 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dc9d05-2034-434b-9582-867bd7156fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:19.808312 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc9d05-2034-434b-9582-867bd7156fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:19.808600 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc9d05-2034-434b-9582-867bd7156fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:19.808881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc9d05-2034-434b-9582-867bd7156fb3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:19.818215 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dc9d05-2034-434b-9582-867bd7156fb3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:19.872868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: WARNING keystone.common.rbac_enforcer.enforcer [None req-70dc9d05-2034-434b-9582-867bd7156fb3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 30 16:12:19.873111 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dc9d05-2034-434b-9582-867bd7156fb3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:19.884283 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 2/4] 10.222.0.67 () {58 vars in 1324 bytes} [Thu Jun 30 16:12:19 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 30 16:12:19.891373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397817fb-ec84-4462-84c6-245bf31f403d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:19.907655 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397817fb-ec84-4462-84c6-245bf31f403d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:19.912196 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f4c60-adf3-49e2-b350-297b0d162e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:19.913351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f4c60-adf3-49e2-b350-297b0d162e31 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:19.913637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f4c60-adf3-49e2-b350-297b0d162e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:19.913913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f4c60-adf3-49e2-b350-297b0d162e31 admin admin] PATH_INFO: `/v3/projects/3967c884c6b647c2a8e60d73c62e7927` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:19.915449 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267f4c60-adf3-49e2-b350-297b0d162e31 admin admin] RBAC: Authorizing `identity:get_project(project_id=3967c884c6b647c2a8e60d73c62e7927)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:19.970557 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: WARNING keystone.common.rbac_enforcer.enforcer [None req-267f4c60-adf3-49e2-b350-297b0d162e31 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 30 16:12:19.971236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267f4c60-adf3-49e2-b350-297b0d162e31 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:19.975773 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 3/5] 10.222.0.67 () {56 vars in 1207 bytes} [Thu Jun 30 16:12:19 2022] GET /identity/v3/projects/3967c884c6b647c2a8e60d73c62e7927 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:22.134393 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859aa6d-e50e-4505-a462-5a884e96508a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:22.134926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859aa6d-e50e-4505-a462-5a884e96508a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:22.134926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3859aa6d-e50e-4505-a462-5a884e96508a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:22.139765 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 3/6] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:12:22.145130 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0e1e4-a8b8-40fd-af1c-9e245a6546a3 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:22.145569 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0e1e4-a8b8-40fd-af1c-9e245a6546a3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:22.145694 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d0e1e4-a8b8-40fd-af1c-9e245a6546a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:22.165411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-13d0e1e4-a8b8-40fd-af1c-9e245a6546a3 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:22.167361 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-13d0e1e4-a8b8-40fd-af1c-9e245a6546a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:22.175333 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 4/7] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 30 16:12:22.197603 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859aa6d-e50e-4505-a462-5a884e96508a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:22.210433 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3859aa6d-e50e-4505-a462-5a884e96508a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:22.215365 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f785c39d-2158-462e-8888-97870cf23523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:22.216092 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785c39d-2158-462e-8888-97870cf23523 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:22.216371 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785c39d-2158-462e-8888-97870cf23523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:22.216644 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f785c39d-2158-462e-8888-97870cf23523 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:22.224393 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785c39d-2158-462e-8888-97870cf23523 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:22.226224 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f785c39d-2158-462e-8888-97870cf23523 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:22.235570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 4/8] 10.222.0.67 () {58 vars in 1324 bytes} [Thu Jun 30 16:12:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 16:12:22.241256 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0e1e4-a8b8-40fd-af1c-9e245a6546a3 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:22.250827 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d0e1e4-a8b8-40fd-af1c-9e245a6546a3 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:22.255225 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbca779-6992-471b-992d-05fe086f3c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:22.256125 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbca779-6992-471b-992d-05fe086f3c4a admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:22.256409 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbca779-6992-471b-992d-05fe086f3c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:22.256717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbca779-6992-471b-992d-05fe086f3c4a admin admin] PATH_INFO: `/v3/users/9242c0a59f6b49568f911aaa0ef688f2` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:22.258120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbca779-6992-471b-992d-05fe086f3c4a admin admin] RBAC: Authorizing `identity:get_user(user_id=9242c0a59f6b49568f911aaa0ef688f2)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:22.260048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbca779-6992-471b-992d-05fe086f3c4a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:22.262236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 5/9] 10.222.0.67 () {56 vars in 1198 bytes} [Thu Jun 30 16:12:22 2022] GET /identity/v3/users/9242c0a59f6b49568f911aaa0ef688f2 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:12:24.373718 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd60269-f2a1-47f6-bf63-3a529ae70563 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:24.374298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd60269-f2a1-47f6-bf63-3a529ae70563 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:24.374298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd60269-f2a1-47f6-bf63-3a529ae70563 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:24.375646 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 5/10] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:24.380834 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a24dfe-ec27-4470-a484-e08841fa7d8f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:24.381338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a24dfe-ec27-4470-a484-e08841fa7d8f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:24.381435 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a24dfe-ec27-4470-a484-e08841fa7d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:24.398640 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e3a24dfe-ec27-4470-a484-e08841fa7d8f None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:24.398842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf7e8c-04b6-40cf-8d84-4a48e7bf451f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:24.399053 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf7e8c-04b6-40cf-8d84-4a48e7bf451f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:24.399335 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf7e8c-04b6-40cf-8d84-4a48e7bf451f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:24.399871 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e3a24dfe-ec27-4470-a484-e08841fa7d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:24.400763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 6/11] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:12:24.406569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422282db-0382-4bf9-926f-f3c4d759a93e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:24.406857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422282db-0382-4bf9-926f-f3c4d759a93e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:24.407134 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422282db-0382-4bf9-926f-f3c4d759a93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:24.448132 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG passlib.handlers.bcrypt [None req-422282db-0382-4bf9-926f-f3c4d759a93e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80297) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 30 16:12:24.448840 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG passlib.handlers.bcrypt [None req-422282db-0382-4bf9-926f-f3c4d759a93e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80297) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 30 16:12:24.481640 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 6/12] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:12:24.495036 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-422282db-0382-4bf9-926f-f3c4d759a93e None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:24.496720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-422282db-0382-4bf9-926f-f3c4d759a93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:24.504698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a24dfe-ec27-4470-a484-e08841fa7d8f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:24.529944 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:24.601330 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 7/13] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:12:24.611992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a24dfe-ec27-4470-a484-e08841fa7d8f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:24.624168 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422282db-0382-4bf9-926f-f3c4d759a93e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:24.647027 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ef5da-172b-420e-8b49-d9c68c24d312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:24.647962 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ef5da-172b-420e-8b49-d9c68c24d312 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:24.648142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ef5da-172b-420e-8b49-d9c68c24d312 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:24.648421 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ef5da-172b-420e-8b49-d9c68c24d312 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:24.660359 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75ef5da-172b-420e-8b49-d9c68c24d312 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:12:24.672458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-b75ef5da-172b-420e-8b49-d9c68c24d312 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:12:24.672458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:12:24.672458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:12:24.672458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:12:24.672458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:12:24.672458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:12:24.672458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:12:24.672458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:12:24.672458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:12:24.672458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:12:24.672458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:12:24.672458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:12:24.672458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:12:24.672458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:12:24.672458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:12:24.672458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:12:24.673521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:12:24.673521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:12:24.673521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:12:24.673521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:12:24.673521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:12:24.673521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:12:24.673521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:12:24.673521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:12:24.673521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:12:24.673521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:12:24.673521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:12:24.673521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:12:24.673521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:12:24.673521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:12:24.673521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:12:24.673521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:12:24.674611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:12:24.674611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:12:24.674611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:12:24.674611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:12:24.674611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:12:24.674611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:12:24.674611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:12:24.674611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:12:24.674611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:12:24.674611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:12:24.674611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:12:24.674611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:12:24.674611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 7/14] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:12:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:12:24.679501 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ef5da-172b-420e-8b49-d9c68c24d312 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:24.731024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422282db-0382-4bf9-926f-f3c4d759a93e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:24.733341 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:24.761137 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:24.767763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82287df1-4712-41b1-8343-b2245ea1a79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:24.768660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82287df1-4712-41b1-8343-b2245ea1a79c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:24.768948 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82287df1-4712-41b1-8343-b2245ea1a79c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:24.769226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82287df1-4712-41b1-8343-b2245ea1a79c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:24.780471 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ef5da-172b-420e-8b49-d9c68c24d312 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:24.815298 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf87aa8-7ef3-40d5-85fb-a7bd42722e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:24.815993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf87aa8-7ef3-40d5-85fb-a7bd42722e83 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:24.816275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf87aa8-7ef3-40d5-85fb-a7bd42722e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:24.816551 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf87aa8-7ef3-40d5-85fb-a7bd42722e83 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:24.817308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf87aa8-7ef3-40d5-85fb-a7bd42722e83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:24.817610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf87aa8-7ef3-40d5-85fb-a7bd42722e83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:24.819019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf87aa8-7ef3-40d5-85fb-a7bd42722e83 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:24.831191 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 8/15] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:12:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:24.836661 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf87aa8-7ef3-40d5-85fb-a7bd42722e83 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:24.838092 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82287df1-4712-41b1-8343-b2245ea1a79c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:24.840270 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82287df1-4712-41b1-8343-b2245ea1a79c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:24.939965 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf87aa8-7ef3-40d5-85fb-a7bd42722e83 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:24.943831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 8/16] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:12:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 16:12:24.950575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82287df1-4712-41b1-8343-b2245ea1a79c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:24.953381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:24.976058 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6dc062-e59b-4a4b-92c6-767b13c18308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:24.976933 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6dc062-e59b-4a4b-92c6-767b13c18308 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:24.977216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6dc062-e59b-4a4b-92c6-767b13c18308 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:24.977487 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6dc062-e59b-4a4b-92c6-767b13c18308 None admin] PATH_INFO: `/v3/users/9242c0a59f6b49568f911aaa0ef688f2` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:24.978684 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6dc062-e59b-4a4b-92c6-767b13c18308 None admin] RBAC: Authorizing `identity:get_user(user_id=9242c0a59f6b49568f911aaa0ef688f2)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:24.980666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6dc062-e59b-4a4b-92c6-767b13c18308 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:24.982296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 9/17] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:12:24 2022] GET /identity/v3/users/9242c0a59f6b49568f911aaa0ef688f2 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:12:24.987772 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6dc062-e59b-4a4b-92c6-767b13c18308 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:25.003530 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:25.052888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82287df1-4712-41b1-8343-b2245ea1a79c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:25.087399 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f1bec-c18f-4e7e-8dd0-797d452b562f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:25.088852 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f1bec-c18f-4e7e-8dd0-797d452b562f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:25.089131 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f1bec-c18f-4e7e-8dd0-797d452b562f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:25.089405 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f1bec-c18f-4e7e-8dd0-797d452b562f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:25.093870 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6dc062-e59b-4a4b-92c6-767b13c18308 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:25.109001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f1bec-c18f-4e7e-8dd0-797d452b562f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:25.110507 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f1bec-c18f-4e7e-8dd0-797d452b562f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:25.127849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579aca55-b4e5-4eec-b288-d4050a0696fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:25.128763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579aca55-b4e5-4eec-b288-d4050a0696fe None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:25.129052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579aca55-b4e5-4eec-b288-d4050a0696fe None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:25.129335 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579aca55-b4e5-4eec-b288-d4050a0696fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:25.130570 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579aca55-b4e5-4eec-b288-d4050a0696fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:25.131137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-5f4f1bec-c18f-4e7e-8dd0-797d452b562f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:12:25.131137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:12:25.131137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:12:25.131137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:12:25.131137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:12:25.131137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:12:25.131137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:12:25.131137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:12:25.131137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:12:25.131137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:12:25.131137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:12:25.131137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:12:25.131137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:12:25.131137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:12:25.131137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:12:25.132069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:12:25.132069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:12:25.132069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:12:25.132069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:12:25.132069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:12:25.132069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:12:25.132069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:12:25.132069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:12:25.132069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:12:25.132069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:12:25.132069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:12:25.132069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:12:25.132069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:12:25.132069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:12:25.132069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:12:25.132069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:12:25.132069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:12:25.132855 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:12:25.132855 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:12:25.132855 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:12:25.132855 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:12:25.133073 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579aca55-b4e5-4eec-b288-d4050a0696fe None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:25.133145 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 9/18] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:12:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:12:25.133656 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 10/19] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:12:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:12:25.139577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f1bec-c18f-4e7e-8dd0-797d452b562f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:25.139726 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579aca55-b4e5-4eec-b288-d4050a0696fe None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:25.198030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:25.222576 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:25.241347 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579aca55-b4e5-4eec-b288-d4050a0696fe None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:25.241747 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f1bec-c18f-4e7e-8dd0-797d452b562f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:25.275363 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088cbed6-fde7-4c53-a9c0-71e3cf3547fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:25.276766 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd2f22-d73f-445c-a6d2-39b69234159c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:25.277044 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cbed6-fde7-4c53-a9c0-71e3cf3547fb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:25.277331 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cbed6-fde7-4c53-a9c0-71e3cf3547fb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:25.277427 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd2f22-d73f-445c-a6d2-39b69234159c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:25.277607 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088cbed6-fde7-4c53-a9c0-71e3cf3547fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:25.277702 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd2f22-d73f-445c-a6d2-39b69234159c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:25.277951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd2f22-d73f-445c-a6d2-39b69234159c None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:25.278445 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088cbed6-fde7-4c53-a9c0-71e3cf3547fb None admin] RBAC: Adding query filter params (role.id=8acc34a163874a60b03728b814eaf075, scope.domain.id=default, user.id=9242c0a59f6b49568f911aaa0ef688f2) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:25.278679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd2f22-d73f-445c-a6d2-39b69234159c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:25.278782 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088cbed6-fde7-4c53-a9c0-71e3cf3547fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:25.278985 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd2f22-d73f-445c-a6d2-39b69234159c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:25.280246 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088cbed6-fde7-4c53-a9c0-71e3cf3547fb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:25.280407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd2f22-d73f-445c-a6d2-39b69234159c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:25.288046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 11/20] 10.222.0.67 () {56 vars in 1322 bytes} [Thu Jun 30 16:12:25 2022] GET /identity/v3/role_assignments?user.id=9242c0a59f6b49568f911aaa0ef688f2&scope.domain.id=default&role.id=8acc34a163874a60b03728b814eaf075 => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:12:25.299101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 10/21] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:27.504056 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea79d5-f45f-4d82-846b-07bbf882553e None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:27.504583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea79d5-f45f-4d82-846b-07bbf882553e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:27.504583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea79d5-f45f-4d82-846b-07bbf882553e None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:27.505966 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 12/22] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:27.511535 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29400cd2-75a1-44b5-b741-736134ac109a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:27.511940 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29400cd2-75a1-44b5-b741-736134ac109a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:27.512169 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29400cd2-75a1-44b5-b741-736134ac109a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:27.533715 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-29400cd2-75a1-44b5-b741-736134ac109a None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:27.534908 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-29400cd2-75a1-44b5-b741-736134ac109a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:27.618431 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 11/23] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:12:27.653537 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea79d5-f45f-4d82-846b-07bbf882553e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:27.797652 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea79d5-f45f-4d82-846b-07bbf882553e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:27.856288 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b5bb4-d7a1-419f-984e-18f25eafd5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:27.857119 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b5bb4-d7a1-419f-984e-18f25eafd5f5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:27.857402 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b5bb4-d7a1-419f-984e-18f25eafd5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:27.857681 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922b5bb4-d7a1-419f-984e-18f25eafd5f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:27.872375 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922b5bb4-d7a1-419f-984e-18f25eafd5f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:12:27.880170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-922b5bb4-d7a1-419f-984e-18f25eafd5f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:12:27.880170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:12:27.880170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:12:27.880170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:12:27.880170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:12:27.880170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:12:27.880170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:12:27.880170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:12:27.880170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:12:27.880170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:12:27.880170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:12:27.880170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:12:27.880170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:12:27.880170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:12:27.880170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:12:27.880170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:12:27.881025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:12:27.881025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:12:27.881025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:12:27.881025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:12:27.881025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:12:27.881025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:12:27.881025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:12:27.881025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:12:27.881025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:12:27.881025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:12:27.881025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:12:27.881025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:12:27.881025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:12:27.881025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:12:27.881025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:12:27.881025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:12:27.882275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:12:27.882275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:12:27.882275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:12:27.882275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:12:27.882275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:12:27.882275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:12:27.882275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:12:27.882275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:12:27.882275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:12:27.882275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:12:27.882275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:12:27.882275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:12:27.882275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 13/24] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:12:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:12:27.888039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29400cd2-75a1-44b5-b741-736134ac109a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:27.988561 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:28.107581 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29400cd2-75a1-44b5-b741-736134ac109a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:28.173058 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a74ae-b563-4733-87d7-d875a2060296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:28.173747 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a74ae-b563-4733-87d7-d875a2060296 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:28.174032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a74ae-b563-4733-87d7-d875a2060296 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:28.174307 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a74ae-b563-4733-87d7-d875a2060296 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:28.175071 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a74ae-b563-4733-87d7-d875a2060296 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:28.175382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a74ae-b563-4733-87d7-d875a2060296 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:28.176812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a74ae-b563-4733-87d7-d875a2060296 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:28.195677 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 12/25] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:12:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:28.201793 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b5bb4-d7a1-419f-984e-18f25eafd5f5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:28.212909 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:28.355948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922b5bb4-d7a1-419f-984e-18f25eafd5f5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:28.363875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8985f6d-ec76-4218-8977-1331e3e8ec5e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:28.364508 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8985f6d-ec76-4218-8977-1331e3e8ec5e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:28.364987 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8985f6d-ec76-4218-8977-1331e3e8ec5e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:28.366739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 13/26] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:12:28.372879 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea069cd-0ff4-45fc-94fa-68e70c797c1a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:28.373400 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea069cd-0ff4-45fc-94fa-68e70c797c1a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:28.373875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea069cd-0ff4-45fc-94fa-68e70c797c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:28.404652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9ea069cd-0ff4-45fc-94fa-68e70c797c1a None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:28.406287 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9ea069cd-0ff4-45fc-94fa-68e70c797c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:28.413126 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d8524-1170-4a2b-ab9e-fb4ab49dbd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:28.414017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d8524-1170-4a2b-ab9e-fb4ab49dbd8e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:28.414300 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d8524-1170-4a2b-ab9e-fb4ab49dbd8e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:28.414574 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d8524-1170-4a2b-ab9e-fb4ab49dbd8e None admin] PATH_INFO: `/v3/users/9242c0a59f6b49568f911aaa0ef688f2` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:28.417984 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371d8524-1170-4a2b-ab9e-fb4ab49dbd8e None admin] RBAC: Authorizing `identity:get_user(user_id=9242c0a59f6b49568f911aaa0ef688f2)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:28.419508 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371d8524-1170-4a2b-ab9e-fb4ab49dbd8e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:28.421497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 14/27] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:12:28 2022] GET /identity/v3/users/9242c0a59f6b49568f911aaa0ef688f2 => generated 247 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:28.426531 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d8524-1170-4a2b-ab9e-fb4ab49dbd8e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:28.546752 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 14/28] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:12:28.548132 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d313e-9055-4b99-9910-c55b8205c5a3 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:28.548420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d313e-9055-4b99-9910-c55b8205c5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:28.548698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d313e-9055-4b99-9910-c55b8205c5a3 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:28.550395 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 15/29] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:28.554054 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:28.555906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ee245-904a-4e17-8db4-4e2c4b4316aa None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:28.556188 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ee245-904a-4e17-8db4-4e2c4b4316aa None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:28.556467 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913ee245-904a-4e17-8db4-4e2c4b4316aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:28.585962 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d8524-1170-4a2b-ab9e-fb4ab49dbd8e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:28.600553 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-913ee245-904a-4e17-8db4-4e2c4b4316aa None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:28.601770 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-913ee245-904a-4e17-8db4-4e2c4b4316aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:28.623882 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3fdf37-1128-445c-9cf0-46c3a22ebee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:28.624607 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fdf37-1128-445c-9cf0-46c3a22ebee5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:28.624891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fdf37-1128-445c-9cf0-46c3a22ebee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:28.625224 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fdf37-1128-445c-9cf0-46c3a22ebee5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:28.626507 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3fdf37-1128-445c-9cf0-46c3a22ebee5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:28.628053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3fdf37-1128-445c-9cf0-46c3a22ebee5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:28.629784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 15/30] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:12:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:28.630581 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3fdf37-1128-445c-9cf0-46c3a22ebee5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:28.665711 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:28.740053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3fdf37-1128-445c-9cf0-46c3a22ebee5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:28.749799 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 16/31] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:12:28.751094 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ee245-904a-4e17-8db4-4e2c4b4316aa None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:28.778449 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b269a9-d1bd-4d8d-ac8a-71f0676e1b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:28.779170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b269a9-d1bd-4d8d-ac8a-71f0676e1b54 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:28.779461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b269a9-d1bd-4d8d-ac8a-71f0676e1b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:28.786154 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b269a9-d1bd-4d8d-ac8a-71f0676e1b54 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:28.786154 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b269a9-d1bd-4d8d-ac8a-71f0676e1b54 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:28.786154 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b269a9-d1bd-4d8d-ac8a-71f0676e1b54 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:28.876761 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913ee245-904a-4e17-8db4-4e2c4b4316aa None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:28.912262 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b93a3-bc8d-4d45-a3e8-931d46c6af64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:28.913135 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b93a3-bc8d-4d45-a3e8-931d46c6af64 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:28.913414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b93a3-bc8d-4d45-a3e8-931d46c6af64 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:28.913719 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b93a3-bc8d-4d45-a3e8-931d46c6af64 None admin] PATH_INFO: `/v3/domains/default/users/9242c0a59f6b49568f911aaa0ef688f2/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:28.916143 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b93a3-bc8d-4d45-a3e8-931d46c6af64 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=9242c0a59f6b49568f911aaa0ef688f2, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:28.917589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b93a3-bc8d-4d45-a3e8-931d46c6af64 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:29.025423 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:29.141139 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 16/32] 10.222.0.67 () {60 vars in 1131 bytes} [Thu Jun 30 16:12:28 2022] POST /identity/v3/roles => generated 219 bytes in 511 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:12:29.143060 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c821ff7-5473-4577-b967-bfb059052cd7 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:29.143705 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c821ff7-5473-4577-b967-bfb059052cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:29.144202 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c821ff7-5473-4577-b967-bfb059052cd7 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:29.146085 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 17/33] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:29.147139 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c821ff7-5473-4577-b967-bfb059052cd7 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:29.184208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 17/34] 10.222.0.67 () {58 vars in 1361 bytes} [Thu Jun 30 16:12:28 2022] PUT /identity/v3/domains/default/users/9242c0a59f6b49568f911aaa0ef688f2/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:12:29.185952 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee5c20-2454-46ed-a4ef-7100509beae0 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:29.186250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee5c20-2454-46ed-a4ef-7100509beae0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:29.186531 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee5c20-2454-46ed-a4ef-7100509beae0 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:29.191801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 18/35] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:29.193355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107f073-c761-4a49-81b4-0106fd0742de None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:29.193648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107f073-c761-4a49-81b4-0106fd0742de None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:29.193934 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107f073-c761-4a49-81b4-0106fd0742de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:29.237016 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f107f073-c761-4a49-81b4-0106fd0742de None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:29.238283 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f107f073-c761-4a49-81b4-0106fd0742de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:29.287871 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c821ff7-5473-4577-b967-bfb059052cd7 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:29.301349 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:29.339759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8312685-ee9b-466d-b32d-4d38af4d09ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:29.340969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8312685-ee9b-466d-b32d-4d38af4d09ad None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:29.341749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8312685-ee9b-466d-b32d-4d38af4d09ad None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:29.342264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8312685-ee9b-466d-b32d-4d38af4d09ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:29.355437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8312685-ee9b-466d-b32d-4d38af4d09ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:29.357456 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8312685-ee9b-466d-b32d-4d38af4d09ad None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:29.373694 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 19/36] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:12:29.376039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8786da8-80dd-4e4f-97d3-56c5a8a7cf67 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:29.376039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8786da8-80dd-4e4f-97d3-56c5a8a7cf67 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:29.376039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8786da8-80dd-4e4f-97d3-56c5a8a7cf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:29.386106 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-e8312685-ee9b-466d-b32d-4d38af4d09ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:12:29.386106 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:12:29.386106 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:12:29.386106 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:12:29.386106 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:12:29.386106 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:12:29.386106 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:12:29.386106 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:12:29.386106 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:12:29.386106 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:12:29.386106 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:12:29.386106 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:12:29.386106 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:12:29.386106 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:12:29.386106 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:12:29.387047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:12:29.387047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:12:29.387047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:12:29.387047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:12:29.387047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:12:29.387047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:12:29.387047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:12:29.387047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:12:29.387047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:12:29.387047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:12:29.387047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:12:29.387047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:12:29.387047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:12:29.387047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:12:29.387047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:12:29.387047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:12:29.387047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:12:29.387870 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:12:29.387870 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:12:29.387870 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:12:29.387870 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:12:29.389893 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 18/37] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:12:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:12:29.396650 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8312685-ee9b-466d-b32d-4d38af4d09ad None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:29.397740 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a8786da8-80dd-4e4f-97d3-56c5a8a7cf67 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:29.399028 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a8786da8-80dd-4e4f-97d3-56c5a8a7cf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:29.484419 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:29.497251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 20/38] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:12:29.498513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8786da8-80dd-4e4f-97d3-56c5a8a7cf67 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:29.609693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:29.623506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8786da8-80dd-4e4f-97d3-56c5a8a7cf67 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:29.643114 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8312685-ee9b-466d-b32d-4d38af4d09ad None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:29.661907 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d8dbb-bbb6-42ef-8319-b141e61187da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:29.662520 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d8dbb-bbb6-42ef-8319-b141e61187da None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:29.662810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d8dbb-bbb6-42ef-8319-b141e61187da None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:29.663093 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d8dbb-bbb6-42ef-8319-b141e61187da None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:29.663883 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d8dbb-bbb6-42ef-8319-b141e61187da None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:29.665403 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d8dbb-bbb6-42ef-8319-b141e61187da None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:29.679276 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fcd3ce-9c93-40b8-907f-ee622aab48ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:29.679968 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcd3ce-9c93-40b8-907f-ee622aab48ad None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:29.685616 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcd3ce-9c93-40b8-907f-ee622aab48ad None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:29.685616 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fcd3ce-9c93-40b8-907f-ee622aab48ad None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:29.685616 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fcd3ce-9c93-40b8-907f-ee622aab48ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:29.685616 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fcd3ce-9c93-40b8-907f-ee622aab48ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:29.685616 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fcd3ce-9c93-40b8-907f-ee622aab48ad None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:29.701140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 19/39] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:29.702497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0152ed8-8db9-460b-be17-88736c45f696 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:29.702794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0152ed8-8db9-460b-be17-88736c45f696 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:29.703076 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0152ed8-8db9-460b-be17-88736c45f696 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:29.704337 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 20/40] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:29.705016 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0152ed8-8db9-460b-be17-88736c45f696 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:29.748272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 21/41] 10.222.0.67 () {60 vars in 1131 bytes} [Thu Jun 30 16:12:29 2022] POST /identity/v3/roles => generated 225 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:12:29.749108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d8dbb-bbb6-42ef-8319-b141e61187da None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:29.845209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0152ed8-8db9-460b-be17-88736c45f696 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:29.871044 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d8dbb-bbb6-42ef-8319-b141e61187da None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:29.879998 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:29.896800 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616643cb-5b74-42cf-bbd5-45baf04d8b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:29.897122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616643cb-5b74-42cf-bbd5-45baf04d8b28 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:29.897366 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616643cb-5b74-42cf-bbd5-45baf04d8b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:29.897715 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616643cb-5b74-42cf-bbd5-45baf04d8b28 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:29.898495 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616643cb-5b74-42cf-bbd5-45baf04d8b28 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:29.900092 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616643cb-5b74-42cf-bbd5-45baf04d8b28 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:29.915440 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d828f-6012-4a4b-898b-bedaa9eba709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:29.916152 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d828f-6012-4a4b-898b-bedaa9eba709 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:29.916938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d828f-6012-4a4b-898b-bedaa9eba709 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:29.917218 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d828f-6012-4a4b-898b-bedaa9eba709 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:29.918507 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d828f-6012-4a4b-898b-bedaa9eba709 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:29.919497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88d828f-6012-4a4b-898b-bedaa9eba709 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:30.047264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 21/42] 10.222.0.67 () {60 vars in 1131 bytes} [Thu Jun 30 16:12:29 2022] POST /identity/v3/roles => generated 223 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:12:30.048587 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadbd06-79c6-4842-8091-a9da019d0fd3 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:30.048871 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadbd06-79c6-4842-8091-a9da019d0fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:30.049154 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadbd06-79c6-4842-8091-a9da019d0fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:30.068020 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9cadbd06-79c6-4842-8091-a9da019d0fd3 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:30.069807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9cadbd06-79c6-4842-8091-a9da019d0fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:30.070900 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:30.089364 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 22/43] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 30 16:12:29 2022] POST /identity/v3/projects => generated 301 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:12:30.168570 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 22/44] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:12:30.194733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d828f-6012-4a4b-898b-bedaa9eba709 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:30.318862 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d828f-6012-4a4b-898b-bedaa9eba709 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:30.358588 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de62c0b-3691-4893-ad56-cb64fa128b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:30.359400 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de62c0b-3691-4893-ad56-cb64fa128b3c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:30.359721 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de62c0b-3691-4893-ad56-cb64fa128b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:30.360007 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de62c0b-3691-4893-ad56-cb64fa128b3c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:30.361182 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de62c0b-3691-4893-ad56-cb64fa128b3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:30.362649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de62c0b-3691-4893-ad56-cb64fa128b3c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:30.364358 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 23/45] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:12:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:30.370286 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadbd06-79c6-4842-8091-a9da019d0fd3 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:30.476382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadbd06-79c6-4842-8091-a9da019d0fd3 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:30.478061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:30.513120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1603813-9126-4f2c-9e3c-1e0b25e9f658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:30.513769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1603813-9126-4f2c-9e3c-1e0b25e9f658 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:30.514052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1603813-9126-4f2c-9e3c-1e0b25e9f658 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:30.514326 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1603813-9126-4f2c-9e3c-1e0b25e9f658 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:30.515024 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1603813-9126-4f2c-9e3c-1e0b25e9f658 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:30.516590 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1603813-9126-4f2c-9e3c-1e0b25e9f658 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:30.647740 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 23/46] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 30 16:12:30 2022] POST /identity/v3/projects => generated 312 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:12:31.431193 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240c2f3-359e-42ea-9a52-9ff384d392e9 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:31.431760 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240c2f3-359e-42ea-9a52-9ff384d392e9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:31.431885 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240c2f3-359e-42ea-9a52-9ff384d392e9 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:31.433261 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 24/47] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:31.438572 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482fa9c-4aea-4293-a9bd-392b11dd662b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:31.438869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482fa9c-4aea-4293-a9bd-392b11dd662b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:31.439163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482fa9c-4aea-4293-a9bd-392b11dd662b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:31.456216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8482fa9c-4aea-4293-a9bd-392b11dd662b None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:31.457436 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8482fa9c-4aea-4293-a9bd-392b11dd662b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:31.524544 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 24/48] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:12:31.549970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240c2f3-359e-42ea-9a52-9ff384d392e9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:31.623010 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:31.658061 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1240c2f3-359e-42ea-9a52-9ff384d392e9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:31.691162 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd8b36-9b5f-401c-aa30-cf45f74c61b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:31.691911 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd8b36-9b5f-401c-aa30-cf45f74c61b1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:31.692193 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd8b36-9b5f-401c-aa30-cf45f74c61b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:31.692465 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd8b36-9b5f-401c-aa30-cf45f74c61b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:31.704732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bd8b36-9b5f-401c-aa30-cf45f74c61b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:12:31.713851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-10bd8b36-9b5f-401c-aa30-cf45f74c61b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:12:31.713851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:12:31.713851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:12:31.713851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:12:31.713851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:12:31.713851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:12:31.713851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:12:31.713851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:12:31.713851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:12:31.713851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:12:31.713851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:12:31.713851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:12:31.713851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:12:31.713851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:12:31.713851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:12:31.713851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:12:31.714679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:12:31.714679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:12:31.714679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:12:31.714679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:12:31.714679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:12:31.714679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:12:31.714679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:12:31.714679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:12:31.714679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:12:31.714679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:12:31.714679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:12:31.714679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:12:31.714679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:12:31.714679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:12:31.714679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:12:31.714679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:12:31.715432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:12:31.715432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:12:31.715432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:12:31.715432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:12:31.715432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:12:31.715432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:12:31.715432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:12:31.715432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:12:31.715432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:12:31.715432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:12:31.715432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:12:31.715432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:12:31.715432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 25/49] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:12:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:12:31.722124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482fa9c-4aea-4293-a9bd-392b11dd662b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:31.808557 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:31.823908 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482fa9c-4aea-4293-a9bd-392b11dd662b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:31.858002 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11009a-e1c8-4c62-8791-69f0efeaa037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:31.858668 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11009a-e1c8-4c62-8791-69f0efeaa037 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:31.858960 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11009a-e1c8-4c62-8791-69f0efeaa037 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:31.859244 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f11009a-e1c8-4c62-8791-69f0efeaa037 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:31.860084 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f11009a-e1c8-4c62-8791-69f0efeaa037 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:31.860352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f11009a-e1c8-4c62-8791-69f0efeaa037 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:31.865493 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f11009a-e1c8-4c62-8791-69f0efeaa037 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:31.870691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 25/50] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:12:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:31.876426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd8b36-9b5f-401c-aa30-cf45f74c61b1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:31.976963 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd8b36-9b5f-401c-aa30-cf45f74c61b1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:32.005149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:32.012766 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970d1afa-c90d-4d75-b6e7-aa0b8d16eed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:32.013644 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d1afa-c90d-4d75-b6e7-aa0b8d16eed6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:32.013932 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d1afa-c90d-4d75-b6e7-aa0b8d16eed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:32.014243 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970d1afa-c90d-4d75-b6e7-aa0b8d16eed6 None admin] PATH_INFO: `/v3/users/9242c0a59f6b49568f911aaa0ef688f2` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:32.015429 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970d1afa-c90d-4d75-b6e7-aa0b8d16eed6 None admin] RBAC: Authorizing `identity:get_user(user_id=9242c0a59f6b49568f911aaa0ef688f2)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:32.022180 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970d1afa-c90d-4d75-b6e7-aa0b8d16eed6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:32.022180 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 26/51] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:12:31 2022] GET /identity/v3/users/9242c0a59f6b49568f911aaa0ef688f2 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:32.023934 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11009a-e1c8-4c62-8791-69f0efeaa037 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:32.125210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f11009a-e1c8-4c62-8791-69f0efeaa037 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:32.158511 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2b6e58-b92f-4f4b-af43-c792b4780828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:32.159311 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b6e58-b92f-4f4b-af43-c792b4780828 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:32.159635 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b6e58-b92f-4f4b-af43-c792b4780828 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:32.159917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2b6e58-b92f-4f4b-af43-c792b4780828 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:32.161107 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b6e58-b92f-4f4b-af43-c792b4780828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:32.162572 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2b6e58-b92f-4f4b-af43-c792b4780828 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:32.164238 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 26/52] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:12:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:32.170713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970d1afa-c90d-4d75-b6e7-aa0b8d16eed6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:32.377884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970d1afa-c90d-4d75-b6e7-aa0b8d16eed6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:32.412009 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90610287-c038-4b8d-bf1d-d402cb2d7fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:32.412665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90610287-c038-4b8d-bf1d-d402cb2d7fbd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:32.412949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90610287-c038-4b8d-bf1d-d402cb2d7fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:32.413226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90610287-c038-4b8d-bf1d-d402cb2d7fbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:32.414054 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90610287-c038-4b8d-bf1d-d402cb2d7fbd None admin] RBAC: Adding query filter params (role.id=8acc34a163874a60b03728b814eaf075, scope.domain.id=default, user.id=9242c0a59f6b49568f911aaa0ef688f2) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:32.414365 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90610287-c038-4b8d-bf1d-d402cb2d7fbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:32.415956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90610287-c038-4b8d-bf1d-d402cb2d7fbd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:32.424530 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 27/53] 10.222.0.67 () {56 vars in 1322 bytes} [Thu Jun 30 16:12:32 2022] GET /identity/v3/role_assignments?user.id=9242c0a59f6b49568f911aaa0ef688f2&scope.domain.id=default&role.id=8acc34a163874a60b03728b814eaf075 => generated 530 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:32.709778 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8bdb1-0d43-4d0d-b59c-6df8a5b308bc None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:32.709979 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8bdb1-0d43-4d0d-b59c-6df8a5b308bc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:32.710258 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8bdb1-0d43-4d0d-b59c-6df8a5b308bc None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:32.711681 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 27/54] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:32.717828 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426450c6-676c-4723-b518-741a6b06fdce None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:32.718269 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426450c6-676c-4723-b518-741a6b06fdce None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:32.718407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426450c6-676c-4723-b518-741a6b06fdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:32.734825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-426450c6-676c-4723-b518-741a6b06fdce None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:32.736095 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-426450c6-676c-4723-b518-741a6b06fdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:32.792228 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:32.802955 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 28/55] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:12:32.827341 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8bdb1-0d43-4d0d-b59c-6df8a5b308bc None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:32.843594 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:32.932881 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8bdb1-0d43-4d0d-b59c-6df8a5b308bc None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:32.966889 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52416369-23b6-44c3-b894-588d458deb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:32.967681 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52416369-23b6-44c3-b894-588d458deb0d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:32.967966 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52416369-23b6-44c3-b894-588d458deb0d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:32.968251 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52416369-23b6-44c3-b894-588d458deb0d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:32.969471 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52416369-23b6-44c3-b894-588d458deb0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:32.970952 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52416369-23b6-44c3-b894-588d458deb0d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:32.972621 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 28/56] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:12:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:32.978289 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426450c6-676c-4723-b518-741a6b06fdce None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:33.078742 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426450c6-676c-4723-b518-741a6b06fdce None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:33.111310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3536dd-033d-4e39-b29a-fcd9db575858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:33.112004 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3536dd-033d-4e39-b29a-fcd9db575858 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:33.112285 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3536dd-033d-4e39-b29a-fcd9db575858 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:33.112562 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3536dd-033d-4e39-b29a-fcd9db575858 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:33.113284 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3536dd-033d-4e39-b29a-fcd9db575858 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:33.114721 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3536dd-033d-4e39-b29a-fcd9db575858 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:33.210874 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 29/57] 10.222.0.67 () {60 vars in 1140 bytes} [Thu Jun 30 16:12:32 2022] POST /identity/v3/projects => generated 298 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:12:35.259915 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeecee9-398f-4a94-923e-f4bfa1b7a292 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:35.260398 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeecee9-398f-4a94-923e-f4bfa1b7a292 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:35.260398 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeecee9-398f-4a94-923e-f4bfa1b7a292 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:35.261755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 29/58] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:35.267326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04a950-98cb-474d-a66d-a3aa74183ca6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:35.267763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04a950-98cb-474d-a66d-a3aa74183ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:35.267931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04a950-98cb-474d-a66d-a3aa74183ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:35.285284 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1d04a950-98cb-474d-a66d-a3aa74183ca6 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:35.286475 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1d04a950-98cb-474d-a66d-a3aa74183ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:35.320938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:35.352063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 30/59] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:12:35.377484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeecee9-398f-4a94-923e-f4bfa1b7a292 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:35.459788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:35.478980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeecee9-398f-4a94-923e-f4bfa1b7a292 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:35.517848 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3563dd0f-882f-4086-8e11-399ecbc13d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:35.518574 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3563dd0f-882f-4086-8e11-399ecbc13d5c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:35.518861 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3563dd0f-882f-4086-8e11-399ecbc13d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:35.519142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3563dd0f-882f-4086-8e11-399ecbc13d5c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:35.520356 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3563dd0f-882f-4086-8e11-399ecbc13d5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:35.521911 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3563dd0f-882f-4086-8e11-399ecbc13d5c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:35.523670 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 30/60] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:12:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:35.530109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04a950-98cb-474d-a66d-a3aa74183ca6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:35.628560 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04a950-98cb-474d-a66d-a3aa74183ca6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:35.662168 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d03375-3025-4187-8873-a41487a72586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:35.662843 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d03375-3025-4187-8873-a41487a72586 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:35.663125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d03375-3025-4187-8873-a41487a72586 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:35.663405 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d03375-3025-4187-8873-a41487a72586 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:35.664151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d03375-3025-4187-8873-a41487a72586 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:35.665608 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d03375-3025-4187-8873-a41487a72586 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:35.776981 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 31/61] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 30 16:12:35 2022] POST /identity/v3/users => generated 275 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:12:38.947918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1727d14-172d-46a1-b6e6-0db0c457c559 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:38.949013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1727d14-172d-46a1-b6e6-0db0c457c559 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:38.949517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1727d14-172d-46a1-b6e6-0db0c457c559 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:38.951122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 31/62] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:38.956557 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90328d-db5d-47b2-bbba-33ca002f2618 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:38.957472 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90328d-db5d-47b2-bbba-33ca002f2618 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:38.957980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90328d-db5d-47b2-bbba-33ca002f2618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:38.988600 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-de90328d-db5d-47b2-bbba-33ca002f2618 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:38.990190 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-de90328d-db5d-47b2-bbba-33ca002f2618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:39.022415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:39.061350 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1853b8d-8983-45e9-8d9b-5c9c7669aa94 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:39.062008 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1853b8d-8983-45e9-8d9b-5c9c7669aa94 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:39.062497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1853b8d-8983-45e9-8d9b-5c9c7669aa94 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:39.064126 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 32/63] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:12:39.071458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033c8eb-8a77-4403-a240-bd4f1142a638 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:39.072179 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033c8eb-8a77-4403-a240-bd4f1142a638 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:39.072775 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033c8eb-8a77-4403-a240-bd4f1142a638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:39.087025 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 32/64] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:12:39.108132 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5033c8eb-8a77-4403-a240-bd4f1142a638 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:39.109455 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5033c8eb-8a77-4403-a240-bd4f1142a638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:39.173646 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90328d-db5d-47b2-bbba-33ca002f2618 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:39.226875 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 33/65] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:12:39.272919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033c8eb-8a77-4403-a240-bd4f1142a638 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:39.330468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90328d-db5d-47b2-bbba-33ca002f2618 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:39.378906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048d5b26-23c3-4594-b3ae-1102bfb692f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:39.380122 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d5b26-23c3-4594-b3ae-1102bfb692f1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:39.380641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d5b26-23c3-4594-b3ae-1102bfb692f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:39.381222 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d5b26-23c3-4594-b3ae-1102bfb692f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:39.394904 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:39.396092 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048d5b26-23c3-4594-b3ae-1102bfb692f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:12:39.407124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-048d5b26-23c3-4594-b3ae-1102bfb692f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:12:39.407124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:12:39.407124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:12:39.407124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:12:39.407124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:12:39.407124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:12:39.407124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:12:39.407124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:12:39.407124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:12:39.407124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:12:39.407124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:12:39.407124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:12:39.407124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:12:39.407124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:12:39.407124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:12:39.407124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:12:39.408257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:12:39.408257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:12:39.408257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:12:39.408257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:12:39.408257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:12:39.408257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:12:39.408257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:12:39.408257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:12:39.408257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:12:39.408257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:12:39.408257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:12:39.408257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:12:39.408257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:12:39.408257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:12:39.408257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:12:39.408257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:12:39.409226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:12:39.409226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:12:39.409226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:12:39.409226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:12:39.409226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:12:39.409226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:12:39.409226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:12:39.409226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:12:39.409226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:12:39.409226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:12:39.409226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:12:39.409226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:12:39.412270 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 33/66] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:12:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:12:39.414329 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bed164-c5d7-4d8b-a7e0-1f01c92512fd None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:39.415042 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bed164-c5d7-4d8b-a7e0-1f01c92512fd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:39.415517 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bed164-c5d7-4d8b-a7e0-1f01c92512fd None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:39.417015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033c8eb-8a77-4403-a240-bd4f1142a638 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:39.417604 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 34/67] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:39.421925 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bed164-c5d7-4d8b-a7e0-1f01c92512fd None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:39.442063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:39.461415 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a84cf7f-6510-45c0-a1aa-598e70fc03f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:39.463242 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84cf7f-6510-45c0-a1aa-598e70fc03f5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:39.463784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84cf7f-6510-45c0-a1aa-598e70fc03f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:39.464260 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a84cf7f-6510-45c0-a1aa-598e70fc03f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:39.478191 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a84cf7f-6510-45c0-a1aa-598e70fc03f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:12:39.491319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-5a84cf7f-6510-45c0-a1aa-598e70fc03f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:12:39.491319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:12:39.491319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:12:39.491319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:12:39.491319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:12:39.491319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:12:39.491319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:12:39.491319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:12:39.491319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:12:39.491319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:12:39.491319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:12:39.491319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:12:39.491319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:12:39.491319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:12:39.491319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:12:39.491319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:12:39.492207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:12:39.492207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:12:39.492207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:12:39.492207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:12:39.492207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:12:39.492207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:12:39.492207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:12:39.492207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:12:39.492207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:12:39.492207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:12:39.492207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:12:39.492207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:12:39.492207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:12:39.492207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:12:39.492207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:12:39.492207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:12:39.492956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:12:39.492956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:12:39.492956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:12:39.492956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:12:39.492956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:12:39.492956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:12:39.492956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:12:39.492956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:12:39.492956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:12:39.492956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:12:39.492956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:12:39.492956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:12:39.495994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 34/68] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 30 16:12:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:12:39.497411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ee0dc-b380-4883-b6b0-7f197450d945 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:39.497852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ee0dc-b380-4883-b6b0-7f197450d945 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:39.498257 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ee0dc-b380-4883-b6b0-7f197450d945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:39.521876 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2a1ee0dc-b380-4883-b6b0-7f197450d945 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:39.523331 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2a1ee0dc-b380-4883-b6b0-7f197450d945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:39.539485 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bed164-c5d7-4d8b-a7e0-1f01c92512fd None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:39.578688 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef0a3d-d9ef-41bb-945f-4dbe7c8652d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:39.579461 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef0a3d-d9ef-41bb-945f-4dbe7c8652d1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:39.579777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef0a3d-d9ef-41bb-945f-4dbe7c8652d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:39.580061 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef0a3d-d9ef-41bb-945f-4dbe7c8652d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:39.580915 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ef0a3d-d9ef-41bb-945f-4dbe7c8652d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:39.581230 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ef0a3d-d9ef-41bb-945f-4dbe7c8652d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:39.582751 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ef0a3d-d9ef-41bb-945f-4dbe7c8652d1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:39.591631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 35/69] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:12:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:39.592664 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef0a3d-d9ef-41bb-945f-4dbe7c8652d1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:39.603736 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 35/70] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:12:39.606039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428a6a7-e8d3-4047-8911-bef7601a2f4a None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:39.606824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428a6a7-e8d3-4047-8911-bef7601a2f4a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:39.607316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428a6a7-e8d3-4047-8911-bef7601a2f4a None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:39.609657 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 36/71] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:39.610567 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428a6a7-e8d3-4047-8911-bef7601a2f4a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:39.680409 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:39.686633 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:39.700135 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef0a3d-d9ef-41bb-945f-4dbe7c8652d1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:39.718304 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428a6a7-e8d3-4047-8911-bef7601a2f4a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:39.734752 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f8253-14f0-4bab-a8ec-c517691a86d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:39.735390 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f8253-14f0-4bab-a8ec-c517691a86d7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:39.735699 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f8253-14f0-4bab-a8ec-c517691a86d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:39.735979 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f8253-14f0-4bab-a8ec-c517691a86d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:39.736734 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f8253-14f0-4bab-a8ec-c517691a86d7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:39.737046 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f8253-14f0-4bab-a8ec-c517691a86d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:39.738446 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4f8253-14f0-4bab-a8ec-c517691a86d7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:39.747996 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 36/72] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:12:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:39.749278 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a3399-50c6-45af-822f-4b45aec7b926 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:39.749559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a3399-50c6-45af-822f-4b45aec7b926 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:39.749841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a3399-50c6-45af-822f-4b45aec7b926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:39.754130 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d4cd31-5bf3-4741-92c5-269a39a840d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:39.755081 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4cd31-5bf3-4741-92c5-269a39a840d3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:39.755365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4cd31-5bf3-4741-92c5-269a39a840d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:39.755674 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d4cd31-5bf3-4741-92c5-269a39a840d3 None admin] PATH_INFO: `/v3/users/3181dca028754df5a6df1154d3635ddd` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:39.766584 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-907a3399-50c6-45af-822f-4b45aec7b926 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:39.767815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-907a3399-50c6-45af-822f-4b45aec7b926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:39.769985 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d4cd31-5bf3-4741-92c5-269a39a840d3 None admin] RBAC: Authorizing `identity:get_user(user_id=3181dca028754df5a6df1154d3635ddd)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:39.771508 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d4cd31-5bf3-4741-92c5-269a39a840d3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:39.773365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 37/73] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:12:39 2022] GET /identity/v3/users/3181dca028754df5a6df1154d3635ddd => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:39.774438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7280d-3dc5-464b-b580-9165cc042d84 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:39.774716 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7280d-3dc5-464b-b580-9165cc042d84 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:39.775022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7280d-3dc5-464b-b580-9165cc042d84 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:39.776547 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 38/74] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:39.779419 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7280d-3dc5-464b-b580-9165cc042d84 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:39.840370 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 37/75] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:12:39.841234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a3399-50c6-45af-822f-4b45aec7b926 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:39.898111 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7280d-3dc5-464b-b580-9165cc042d84 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:39.914894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:39.925427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:39.933874 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81f9fc-ce76-4d38-bc65-b7b8323d6058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:39.934668 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81f9fc-ce76-4d38-bc65-b7b8323d6058 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:39.940038 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81f9fc-ce76-4d38-bc65-b7b8323d6058 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:39.940038 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81f9fc-ce76-4d38-bc65-b7b8323d6058 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:39.945127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81f9fc-ce76-4d38-bc65-b7b8323d6058 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:12:39.945282 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a3399-50c6-45af-822f-4b45aec7b926 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:39.952992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ad81f9fc-ce76-4d38-bc65-b7b8323d6058 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:12:39.952992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:12:39.952992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:12:39.952992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:12:39.952992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:12:39.952992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:12:39.952992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:12:39.952992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:12:39.952992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:12:39.952992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:12:39.952992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:12:39.952992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:12:39.952992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:12:39.952992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:12:39.952992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:12:39.952992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:12:39.954285 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:12:39.954285 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:12:39.954285 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:12:39.954285 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:12:39.954285 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:12:39.954285 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:12:39.954285 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:12:39.954285 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:12:39.954285 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:12:39.954285 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:12:39.954285 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:12:39.954285 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:12:39.954285 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:12:39.954285 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:12:39.954285 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:12:39.954285 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:12:39.955015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:12:39.955015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:12:39.955015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:12:39.955015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:12:39.955015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:12:39.955015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:12:39.955015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:12:39.955015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:12:39.955015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:12:39.955015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:12:39.955015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:12:39.955015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:12:39.955015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 39/76] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:12:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:12:39.955015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81f9fc-ce76-4d38-bc65-b7b8323d6058 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:39.984917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb29796-9893-449b-bab0-174743c3f64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:39.985768 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb29796-9893-449b-bab0-174743c3f64d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:39.986058 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb29796-9893-449b-bab0-174743c3f64d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:39.986338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb29796-9893-449b-bab0-174743c3f64d None admin] PATH_INFO: `/v3/users/3181dca028754df5a6df1154d3635ddd` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:39.987586 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb29796-9893-449b-bab0-174743c3f64d None admin] RBAC: Authorizing `identity:get_user(user_id=3181dca028754df5a6df1154d3635ddd)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:39.989107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb29796-9893-449b-bab0-174743c3f64d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:39.990746 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 38/77] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:12:39 2022] GET /identity/v3/users/3181dca028754df5a6df1154d3635ddd => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:39.995411 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929c5c3-7898-4c4c-93cd-95d0a89cceb0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:39.995411 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929c5c3-7898-4c4c-93cd-95d0a89cceb0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:39.995411 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929c5c3-7898-4c4c-93cd-95d0a89cceb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:40.011702 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a929c5c3-7898-4c4c-93cd-95d0a89cceb0 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:40.012925 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a929c5c3-7898-4c4c-93cd-95d0a89cceb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:40.059733 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81f9fc-ce76-4d38-bc65-b7b8323d6058 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:40.081877 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 39/78] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:12:40.082724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929c5c3-7898-4c4c-93cd-95d0a89cceb0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:40.108371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa801d-4217-464b-81f5-1f4a6e9eff94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:40.109025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa801d-4217-464b-81f5-1f4a6e9eff94 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:40.109315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa801d-4217-464b-81f5-1f4a6e9eff94 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:40.109599 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aa801d-4217-464b-81f5-1f4a6e9eff94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:40.150618 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:40.164976 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:40.178141 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aa801d-4217-464b-81f5-1f4a6e9eff94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:40.179668 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aa801d-4217-464b-81f5-1f4a6e9eff94 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:40.180776 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929c5c3-7898-4c4c-93cd-95d0a89cceb0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:40.216247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee150c-35dc-4074-80ba-f6971648fcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:40.217046 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee150c-35dc-4074-80ba-f6971648fcda None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:40.217332 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee150c-35dc-4074-80ba-f6971648fcda None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:40.217610 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee150c-35dc-4074-80ba-f6971648fcda None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:40.228708 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee150c-35dc-4074-80ba-f6971648fcda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:12:40.236556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-36ee150c-35dc-4074-80ba-f6971648fcda None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:12:40.236556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:12:40.236556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:12:40.236556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:12:40.236556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:12:40.236556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:12:40.236556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:12:40.236556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:12:40.236556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:12:40.236556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:12:40.236556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:12:40.236556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:12:40.236556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:12:40.236556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:12:40.236556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:12:40.236556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:12:40.237470 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:12:40.237470 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:12:40.237470 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:12:40.237470 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:12:40.237470 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:12:40.237470 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:12:40.237470 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:12:40.237470 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:12:40.237470 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:12:40.237470 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:12:40.237470 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:12:40.237470 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:12:40.237470 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:12:40.237470 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:12:40.237470 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:12:40.237470 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:12:40.238213 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:12:40.238213 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:12:40.238213 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:12:40.238213 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:12:40.238213 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:12:40.238213 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:12:40.238213 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:12:40.238213 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:12:40.238213 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:12:40.238213 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:12:40.238213 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:12:40.238213 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:12:40.238213 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 40/79] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:12:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:12:40.238838 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee150c-35dc-4074-80ba-f6971648fcda None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:40.286060 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 40/80] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:12:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:12:40.286915 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa801d-4217-464b-81f5-1f4a6e9eff94 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:40.343143 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee150c-35dc-4074-80ba-f6971648fcda None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:40.373612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:40.376488 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:40.382515 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a41477-789b-4597-a351-9a412a144cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:40.386325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a41477-789b-4597-a351-9a412a144cca None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:40.386325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a41477-789b-4597-a351-9a412a144cca None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:40.386325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a41477-789b-4597-a351-9a412a144cca None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:40.386325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a41477-789b-4597-a351-9a412a144cca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:40.386325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a41477-789b-4597-a351-9a412a144cca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:40.386740 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aa801d-4217-464b-81f5-1f4a6e9eff94 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:40.386823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a41477-789b-4597-a351-9a412a144cca None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:40.395007 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 41/81] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:12:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:40.395833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a41477-789b-4597-a351-9a412a144cca None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:40.422326 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110e4e0-9e61-44d7-bc47-4355ebb51894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:40.422978 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110e4e0-9e61-44d7-bc47-4355ebb51894 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:40.423261 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110e4e0-9e61-44d7-bc47-4355ebb51894 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:40.423635 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110e4e0-9e61-44d7-bc47-4355ebb51894 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:40.491924 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110e4e0-9e61-44d7-bc47-4355ebb51894 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:40.493364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110e4e0-9e61-44d7-bc47-4355ebb51894 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:40.500160 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a41477-789b-4597-a351-9a412a144cca None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:40.535346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b98c1-8690-49b6-8043-4f765f492f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:40.536180 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b98c1-8690-49b6-8043-4f765f492f8f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:40.536471 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b98c1-8690-49b6-8043-4f765f492f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:40.536754 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b98c1-8690-49b6-8043-4f765f492f8f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:40.537994 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403b98c1-8690-49b6-8043-4f765f492f8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:40.539460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403b98c1-8690-49b6-8043-4f765f492f8f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:40.541791 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 42/82] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:12:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:40.542563 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b98c1-8690-49b6-8043-4f765f492f8f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:40.584773 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:40.597651 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 41/83] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:12:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:12:40.598476 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110e4e0-9e61-44d7-bc47-4355ebb51894 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:40.606404 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:40.645503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b98c1-8690-49b6-8043-4f765f492f8f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:40.680470 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ac18e-e237-40af-a6ea-ae7c105cca48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:40.681177 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ac18e-e237-40af-a6ea-ae7c105cca48 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:40.681426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ac18e-e237-40af-a6ea-ae7c105cca48 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:40.681703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ac18e-e237-40af-a6ea-ae7c105cca48 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:40.682452 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ac18e-e237-40af-a6ea-ae7c105cca48 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:40.682761 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ac18e-e237-40af-a6ea-ae7c105cca48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:40.684302 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ac18e-e237-40af-a6ea-ae7c105cca48 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:40.692494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 43/84] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:12:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:40.693290 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ac18e-e237-40af-a6ea-ae7c105cca48 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:40.703592 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110e4e0-9e61-44d7-bc47-4355ebb51894 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:40.738611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5751465-c18e-47c7-a091-17024ed54f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:40.739490 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5751465-c18e-47c7-a091-17024ed54f3c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:40.739802 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5751465-c18e-47c7-a091-17024ed54f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:40.740092 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5751465-c18e-47c7-a091-17024ed54f3c None admin] PATH_INFO: `/v3/projects/e54d8744be3c4672b24fa2e19f024af0` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:40.741335 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5751465-c18e-47c7-a091-17024ed54f3c None admin] RBAC: Authorizing `identity:get_project(project_id=e54d8744be3c4672b24fa2e19f024af0)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:40.742870 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5751465-c18e-47c7-a091-17024ed54f3c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:40.744800 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 42/85] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:12:40 2022] GET /identity/v3/projects/e54d8744be3c4672b24fa2e19f024af0 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:40.745715 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5751465-c18e-47c7-a091-17024ed54f3c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:40.797767 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ac18e-e237-40af-a6ea-ae7c105cca48 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:40.804141 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:40.820754 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:40.832450 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246840f-53eb-40ee-b208-218f8aa81a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:40.833382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246840f-53eb-40ee-b208-218f8aa81a0f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:40.833659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246840f-53eb-40ee-b208-218f8aa81a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:40.833930 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246840f-53eb-40ee-b208-218f8aa81a0f None admin] PATH_INFO: `/v3/users/9242c0a59f6b49568f911aaa0ef688f2` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:40.835109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246840f-53eb-40ee-b208-218f8aa81a0f None admin] RBAC: Authorizing `identity:get_user(user_id=9242c0a59f6b49568f911aaa0ef688f2)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:40.836599 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246840f-53eb-40ee-b208-218f8aa81a0f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:40.839021 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 44/86] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:12:40 2022] GET /identity/v3/users/9242c0a59f6b49568f911aaa0ef688f2 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:40.839816 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246840f-53eb-40ee-b208-218f8aa81a0f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:40.853428 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5751465-c18e-47c7-a091-17024ed54f3c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:40.887273 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5275dd-ec39-4823-8d7c-9410801d8ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:40.887981 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5275dd-ec39-4823-8d7c-9410801d8ac7 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:40.888269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5275dd-ec39-4823-8d7c-9410801d8ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:40.888545 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5275dd-ec39-4823-8d7c-9410801d8ac7 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:40.889721 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5275dd-ec39-4823-8d7c-9410801d8ac7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:40.890778 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5275dd-ec39-4823-8d7c-9410801d8ac7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:40.937963 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246840f-53eb-40ee-b208-218f8aa81a0f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:40.971206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e062e-0608-40d4-bc7a-c759a3eed34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:40.972096 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e062e-0608-40d4-bc7a-c759a3eed34d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:40.972390 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e062e-0608-40d4-bc7a-c759a3eed34d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:40.972668 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565e062e-0608-40d4-bc7a-c759a3eed34d None admin] PATH_INFO: `/v3/projects/e54d8744be3c4672b24fa2e19f024af0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:40.973848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e062e-0608-40d4-bc7a-c759a3eed34d None admin] RBAC: Authorizing `identity:get_project(project_id=e54d8744be3c4672b24fa2e19f024af0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:40.975289 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565e062e-0608-40d4-bc7a-c759a3eed34d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:40.977002 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 45/87] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:12:40 2022] GET /identity/v3/projects/e54d8744be3c4672b24fa2e19f024af0 => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:40.977771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e062e-0608-40d4-bc7a-c759a3eed34d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:40.993872 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 43/88] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 30 16:12:40 2022] POST /identity/v3/users => generated 289 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:12:40.993872 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5275dd-ec39-4823-8d7c-9410801d8ac7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:41.058277 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:41.110428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565e062e-0608-40d4-bc7a-c759a3eed34d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:41.119365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5275dd-ec39-4823-8d7c-9410801d8ac7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:41.133832 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:41.153499 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b80ef9-a403-40b1-87b7-0c14ad1addf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:41.154494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b80ef9-a403-40b1-87b7-0c14ad1addf2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:41.154786 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b80ef9-a403-40b1-87b7-0c14ad1addf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:41.155068 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b80ef9-a403-40b1-87b7-0c14ad1addf2 None admin] PATH_INFO: `/v3/users/3181dca028754df5a6df1154d3635ddd` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:41.156362 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b80ef9-a403-40b1-87b7-0c14ad1addf2 None admin] RBAC: Authorizing `identity:get_user(user_id=3181dca028754df5a6df1154d3635ddd)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:41.157930 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b80ef9-a403-40b1-87b7-0c14ad1addf2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:41.160218 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 46/89] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:12:40 2022] GET /identity/v3/users/3181dca028754df5a6df1154d3635ddd => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:41.161000 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24064e7-988c-48da-aa81-98853def5977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:41.161153 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b80ef9-a403-40b1-87b7-0c14ad1addf2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:41.161729 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24064e7-988c-48da-aa81-98853def5977 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:41.162024 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24064e7-988c-48da-aa81-98853def5977 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:41.162308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24064e7-988c-48da-aa81-98853def5977 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:41.163201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24064e7-988c-48da-aa81-98853def5977 None admin] RBAC: Adding query filter params (role.id=9ab3e302ead54950b7c30103f1e7fda7, scope.project.id=e54d8744be3c4672b24fa2e19f024af0, user.id=3181dca028754df5a6df1154d3635ddd) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:41.163582 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24064e7-988c-48da-aa81-98853def5977 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:41.165386 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24064e7-988c-48da-aa81-98853def5977 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:41.171936 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 44/90] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:12:40 2022] GET /identity/v3/role_assignments?user.id=3181dca028754df5a6df1154d3635ddd&scope.project.id=e54d8744be3c4672b24fa2e19f024af0&role.id=9ab3e302ead54950b7c30103f1e7fda7 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:41.172916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24064e7-988c-48da-aa81-98853def5977 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:41.315422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b80ef9-a403-40b1-87b7-0c14ad1addf2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:41.320994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24064e7-988c-48da-aa81-98853def5977 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:41.357171 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:41.367332 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b568ff-f0ce-48b1-aeb7-68bec52fcf55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:41.368296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b568ff-f0ce-48b1-aeb7-68bec52fcf55 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:41.373282 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b568ff-f0ce-48b1-aeb7-68bec52fcf55 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:41.373282 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b568ff-f0ce-48b1-aeb7-68bec52fcf55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:41.377629 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4de04-8786-4203-8534-fa37f136a370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:41.378308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4de04-8786-4203-8534-fa37f136a370 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:41.378597 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4de04-8786-4203-8534-fa37f136a370 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:41.378878 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4de04-8786-4203-8534-fa37f136a370 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:41.379761 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4de04-8786-4203-8534-fa37f136a370 None admin] RBAC: Adding query filter params (role.id=cfeec012f4ec48138c0e637e9a7fde61, scope.project.id=e54d8744be3c4672b24fa2e19f024af0, user.id=3181dca028754df5a6df1154d3635ddd) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:41.380081 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4de04-8786-4203-8534-fa37f136a370 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:41.382179 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4de04-8786-4203-8534-fa37f136a370 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:41.391185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 45/91] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:12:41 2022] GET /identity/v3/role_assignments?user.id=3181dca028754df5a6df1154d3635ddd&scope.project.id=e54d8744be3c4672b24fa2e19f024af0&role.id=cfeec012f4ec48138c0e637e9a7fde61 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:41.392228 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4de04-8786-4203-8534-fa37f136a370 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:41.453362 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:41.528043 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4de04-8786-4203-8534-fa37f136a370 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:41.560695 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b568ff-f0ce-48b1-aeb7-68bec52fcf55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:41.562219 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b568ff-f0ce-48b1-aeb7-68bec52fcf55 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:41.576057 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647a10a-82dc-497f-a0f3-c7ea6c51691c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:41.577035 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647a10a-82dc-497f-a0f3-c7ea6c51691c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:41.577497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647a10a-82dc-497f-a0f3-c7ea6c51691c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:41.577947 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647a10a-82dc-497f-a0f3-c7ea6c51691c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:41.660626 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c647a10a-82dc-497f-a0f3-c7ea6c51691c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:41.662432 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c647a10a-82dc-497f-a0f3-c7ea6c51691c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:41.711963 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:41.833604 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 47/92] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:12:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 673 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:12:41.834355 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 46/93] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:12:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:12:41.839564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b568ff-f0ce-48b1-aeb7-68bec52fcf55 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:41.843382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647a10a-82dc-497f-a0f3-c7ea6c51691c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:41.904061 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:42.063615 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b568ff-f0ce-48b1-aeb7-68bec52fcf55 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:42.107779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647a10a-82dc-497f-a0f3-c7ea6c51691c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:42.135918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b529832-ac3d-4638-95d8-bf8bf8a4fad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:42.136792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b529832-ac3d-4638-95d8-bf8bf8a4fad3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:42.137078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b529832-ac3d-4638-95d8-bf8bf8a4fad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:42.137359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b529832-ac3d-4638-95d8-bf8bf8a4fad3 None admin] PATH_INFO: `/v3/projects/e54d8744be3c4672b24fa2e19f024af0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:42.139897 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b529832-ac3d-4638-95d8-bf8bf8a4fad3 None admin] RBAC: Authorizing `identity:get_project(project_id=e54d8744be3c4672b24fa2e19f024af0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:42.141406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b529832-ac3d-4638-95d8-bf8bf8a4fad3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:42.147232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 48/94] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:12:41 2022] GET /identity/v3/projects/e54d8744be3c4672b24fa2e19f024af0 => generated 298 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:42.167065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d521893-54d6-4575-ba3e-b65f81ceea30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:42.167997 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d521893-54d6-4575-ba3e-b65f81ceea30 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:42.168319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d521893-54d6-4575-ba3e-b65f81ceea30 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:42.168606 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d521893-54d6-4575-ba3e-b65f81ceea30 None admin] PATH_INFO: `/v3/projects/54d37e8eb5424296b3a4c9602e06061d` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:42.169871 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d521893-54d6-4575-ba3e-b65f81ceea30 None admin] RBAC: Authorizing `identity:get_project(project_id=54d37e8eb5424296b3a4c9602e06061d)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:42.171363 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d521893-54d6-4575-ba3e-b65f81ceea30 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:42.173297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b529832-ac3d-4638-95d8-bf8bf8a4fad3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:42.178044 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 47/95] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:12:41 2022] GET /identity/v3/projects/54d37e8eb5424296b3a4c9602e06061d => generated 312 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:12:42.184190 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d521893-54d6-4575-ba3e-b65f81ceea30 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:42.206656 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:42.355037 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b529832-ac3d-4638-95d8-bf8bf8a4fad3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:42.371892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:42.379709 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d521893-54d6-4575-ba3e-b65f81ceea30 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:42.422650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e08f73-4544-4371-9840-a338b576bd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:42.423329 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e08f73-4544-4371-9840-a338b576bd70 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:42.423643 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e08f73-4544-4371-9840-a338b576bd70 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:42.423926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e08f73-4544-4371-9840-a338b576bd70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:42.424772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e08f73-4544-4371-9840-a338b576bd70 None admin] RBAC: Adding query filter params (role.id=8acc34a163874a60b03728b814eaf075, scope.project.id=e54d8744be3c4672b24fa2e19f024af0, user.id=9242c0a59f6b49568f911aaa0ef688f2) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:42.425096 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e08f73-4544-4371-9840-a338b576bd70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:42.430809 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e08f73-4544-4371-9840-a338b576bd70 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:42.433450 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 49/96] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:12:42 2022] GET /identity/v3/role_assignments?user.id=9242c0a59f6b49568f911aaa0ef688f2&scope.project.id=e54d8744be3c4672b24fa2e19f024af0&role.id=8acc34a163874a60b03728b814eaf075 => generated 259 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:12:42.453152 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a5e7d-4f0e-40fd-9553-9238d9aadd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:42.453826 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a5e7d-4f0e-40fd-9553-9238d9aadd46 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:42.454124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a5e7d-4f0e-40fd-9553-9238d9aadd46 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:42.454405 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a5e7d-4f0e-40fd-9553-9238d9aadd46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:42.455257 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a5e7d-4f0e-40fd-9553-9238d9aadd46 None admin] RBAC: Adding query filter params (role.id=9ab3e302ead54950b7c30103f1e7fda7, scope.project.id=54d37e8eb5424296b3a4c9602e06061d, user.id=3181dca028754df5a6df1154d3635ddd) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:42.455668 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a5e7d-4f0e-40fd-9553-9238d9aadd46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:42.457140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a5e7d-4f0e-40fd-9553-9238d9aadd46 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:42.467937 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 48/97] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:12:42 2022] GET /identity/v3/role_assignments?user.id=3181dca028754df5a6df1154d3635ddd&scope.project.id=54d37e8eb5424296b3a4c9602e06061d&role.id=9ab3e302ead54950b7c30103f1e7fda7 => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:43.592632 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7b088-a28c-42d9-a998-79a9aac1a688 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:43.593649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7b088-a28c-42d9-a998-79a9aac1a688 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:43.594204 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7b088-a28c-42d9-a998-79a9aac1a688 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:43.595819 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 50/98] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:43.601307 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4f70e-7455-4edd-b10d-06f80d02ba5f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:43.601739 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4f70e-7455-4edd-b10d-06f80d02ba5f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:43.601907 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4f70e-7455-4edd-b10d-06f80d02ba5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:43.619985 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-41e4f70e-7455-4edd-b10d-06f80d02ba5f None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:43.621590 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-41e4f70e-7455-4edd-b10d-06f80d02ba5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:43.704184 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 49/99] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:12:43.738941 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af7b088-a28c-42d9-a998-79a9aac1a688 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:43.865571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af7b088-a28c-42d9-a998-79a9aac1a688 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:43.903119 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e76a9d-b1a9-4bcc-81c6-2dbed9d0c696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:43.904346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e76a9d-b1a9-4bcc-81c6-2dbed9d0c696 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:43.904861 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e76a9d-b1a9-4bcc-81c6-2dbed9d0c696 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:43.905340 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e76a9d-b1a9-4bcc-81c6-2dbed9d0c696 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:43.917950 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e76a9d-b1a9-4bcc-81c6-2dbed9d0c696 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:12:43.926570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-68e76a9d-b1a9-4bcc-81c6-2dbed9d0c696 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:12:43.926570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:12:43.926570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:12:43.926570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:12:43.926570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:12:43.926570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:12:43.926570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:12:43.926570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:12:43.926570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:12:43.926570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:12:43.926570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:12:43.926570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:12:43.926570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:12:43.926570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:12:43.926570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:12:43.926570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:12:43.927415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:12:43.927415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:12:43.927415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:12:43.927415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:12:43.927415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:12:43.927415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:12:43.927415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:12:43.927415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:12:43.927415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:12:43.927415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:12:43.927415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:12:43.927415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:12:43.927415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:12:43.927415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:12:43.927415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:12:43.927415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:12:43.928267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:12:43.928267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:12:43.928267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:12:43.928267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:12:43.928267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:12:43.928267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:12:43.928267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:12:43.928267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:12:43.928267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:12:43.928267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:12:43.928267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:12:43.928267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:12:43.931647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 51/100] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:12:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:12:43.937112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4f70e-7455-4edd-b10d-06f80d02ba5f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:43.950418 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:44.088881 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4f70e-7455-4edd-b10d-06f80d02ba5f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:44.137373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e729d-864a-4e54-8386-19b5a8543d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:44.138044 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e729d-864a-4e54-8386-19b5a8543d06 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:44.138332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e729d-864a-4e54-8386-19b5a8543d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:44.138614 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e729d-864a-4e54-8386-19b5a8543d06 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:44.139375 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e729d-864a-4e54-8386-19b5a8543d06 None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:44.139706 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e729d-864a-4e54-8386-19b5a8543d06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:44.141129 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e729d-864a-4e54-8386-19b5a8543d06 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:44.151264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 50/101] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:12:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:44.156801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e76a9d-b1a9-4bcc-81c6-2dbed9d0c696 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:44.176994 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:44.236578 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c6297-54c9-4b31-859a-350cb79c038d None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:44.236913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c6297-54c9-4b31-859a-350cb79c038d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:44.237147 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c6297-54c9-4b31-859a-350cb79c038d None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:44.238905 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 51/102] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:12:44.245938 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e14ffa-cb1d-4f65-965c-75700a2be0bd None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:44.246234 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e14ffa-cb1d-4f65-965c-75700a2be0bd None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:44.246596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e14ffa-cb1d-4f65-965c-75700a2be0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:44.269006 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-17e14ffa-cb1d-4f65-965c-75700a2be0bd None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:44.270256 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-17e14ffa-cb1d-4f65-965c-75700a2be0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:44.322819 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e76a9d-b1a9-4bcc-81c6-2dbed9d0c696 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:44.381072 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:44.390830 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e23950-c599-4559-88bf-4cced61314cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:44.390830 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e23950-c599-4559-88bf-4cced61314cc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:44.390830 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e23950-c599-4559-88bf-4cced61314cc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:44.390830 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e23950-c599-4559-88bf-4cced61314cc None admin] PATH_INFO: `/v3/users/3181dca028754df5a6df1154d3635ddd` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:44.391167 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e23950-c599-4559-88bf-4cced61314cc None admin] RBAC: Authorizing `identity:get_user(user_id=3181dca028754df5a6df1154d3635ddd)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:44.392638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e23950-c599-4559-88bf-4cced61314cc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:44.394448 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 52/103] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:12:44 2022] GET /identity/v3/users/3181dca028754df5a6df1154d3635ddd => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:44.402322 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e23950-c599-4559-88bf-4cced61314cc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:44.404103 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 52/104] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:12:44.423170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b4d70-ceea-4bb4-a874-884bcb08ad9e None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:44.423804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b4d70-ceea-4bb4-a874-884bcb08ad9e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:44.424303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81b4d70-ceea-4bb4-a874-884bcb08ad9e None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:44.426056 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 53/105] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:12:44.432055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f390f2-9dab-49cc-881e-096b9af312e1 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:44.432583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f390f2-9dab-49cc-881e-096b9af312e1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:44.433093 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f390f2-9dab-49cc-881e-096b9af312e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:44.456500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-39f390f2-9dab-49cc-881e-096b9af312e1 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:44.457727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-39f390f2-9dab-49cc-881e-096b9af312e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:44.519849 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:44.529767 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e23950-c599-4559-88bf-4cced61314cc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:44.574983 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681a114-f3e7-4bb1-ac38-a7bf24315e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:44.576362 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681a114-f3e7-4bb1-ac38-a7bf24315e0e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:44.576990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681a114-f3e7-4bb1-ac38-a7bf24315e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:44.577445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681a114-f3e7-4bb1-ac38-a7bf24315e0e None admin] PATH_INFO: `/v3/projects/e54d8744be3c4672b24fa2e19f024af0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:44.579093 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681a114-f3e7-4bb1-ac38-a7bf24315e0e None admin] RBAC: Authorizing `identity:get_project(project_id=e54d8744be3c4672b24fa2e19f024af0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:44.580794 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681a114-f3e7-4bb1-ac38-a7bf24315e0e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:44.582676 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 53/106] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:12:44 2022] GET /identity/v3/projects/e54d8744be3c4672b24fa2e19f024af0 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:44.583642 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681a114-f3e7-4bb1-ac38-a7bf24315e0e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:44.633740 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 54/107] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:12:44.635652 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f390f2-9dab-49cc-881e-096b9af312e1 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:44.731320 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681a114-f3e7-4bb1-ac38-a7bf24315e0e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:44.804559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f390f2-9dab-49cc-881e-096b9af312e1 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:44.821121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcab07-5a8b-4578-8fd1-54087a057b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:44.822494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcab07-5a8b-4578-8fd1-54087a057b46 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:44.822790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcab07-5a8b-4578-8fd1-54087a057b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:44.823068 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dcab07-5a8b-4578-8fd1-54087a057b46 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:44.833202 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:44.847763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155ec61-0425-4f74-ba8e-772cb97300a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:44.854172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155ec61-0425-4f74-ba8e-772cb97300a8 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:44.854172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155ec61-0425-4f74-ba8e-772cb97300a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:44.854172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155ec61-0425-4f74-ba8e-772cb97300a8 None admin] PATH_INFO: `/v3/projects/e54d8744be3c4672b24fa2e19f024af0/users/3181dca028754df5a6df1154d3635ddd/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:44.854448 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dcab07-5a8b-4578-8fd1-54087a057b46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:12:44.858373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9155ec61-0425-4f74-ba8e-772cb97300a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=e54d8744be3c4672b24fa2e19f024af0, user_id=3181dca028754df5a6df1154d3635ddd, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:44.860045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9155ec61-0425-4f74-ba8e-772cb97300a8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:44.870654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e2dcab07-5a8b-4578-8fd1-54087a057b46 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 16:12:44.870654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:12:44.870654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:12:44.870654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:12:44.870654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:12:44.870654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:12:44.870654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:12:44.870654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:12:44.870654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:12:44.870654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:12:44.870654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:12:44.870654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:12:44.870654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:12:44.870654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:12:44.870654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:12:44.870654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:12:44.871755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:12:44.871755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:12:44.871755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:12:44.871755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:12:44.871755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:12:44.871755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:12:44.871755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:12:44.871755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:12:44.871755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:12:44.871755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:12:44.871755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:12:44.871755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:12:44.871755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:12:44.871755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:12:44.871755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:12:44.871755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:12:44.872506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:12:44.872506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:12:44.872506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:12:44.872506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:12:44.872506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:12:44.872506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:12:44.872506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:12:44.872506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:12:44.872506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:12:44.872506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:12:44.872506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 16:12:44.872506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:12:44.874967 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 54/108] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:12:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:12:44.876044 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcab07-5a8b-4578-8fd1-54087a057b46 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:44.895925 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:44.988439 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dcab07-5a8b-4578-8fd1-54087a057b46 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:44.992449 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 55/109] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 30 16:12:44 2022] PUT /identity/v3/projects/e54d8744be3c4672b24fa2e19f024af0/users/3181dca028754df5a6df1154d3635ddd/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:12:44.993855 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155ec61-0425-4f74-ba8e-772cb97300a8 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:45.028014 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b4e8cc-62cf-4bee-a3b5-108a2816897f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:45.028877 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4e8cc-62cf-4bee-a3b5-108a2816897f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:45.029209 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4e8cc-62cf-4bee-a3b5-108a2816897f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:45.029492 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4e8cc-62cf-4bee-a3b5-108a2816897f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:45.030810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b4e8cc-62cf-4bee-a3b5-108a2816897f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:45.032503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b4e8cc-62cf-4bee-a3b5-108a2816897f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:45.034679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 55/110] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:12:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:45.040906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b4e8cc-62cf-4bee-a3b5-108a2816897f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:45.139444 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:45.160576 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b4e8cc-62cf-4bee-a3b5-108a2816897f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:45.163137 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155ec61-0425-4f74-ba8e-772cb97300a8 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:45.200723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8570b02-f046-4647-b25f-03dec0c82266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:45.200924 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5516e813-8fcc-4ad3-bcb7-8ebb72df606c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:45.201405 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8570b02-f046-4647-b25f-03dec0c82266 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:45.201598 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5516e813-8fcc-4ad3-bcb7-8ebb72df606c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:45.201697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8570b02-f046-4647-b25f-03dec0c82266 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:45.201880 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5516e813-8fcc-4ad3-bcb7-8ebb72df606c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:45.201973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8570b02-f046-4647-b25f-03dec0c82266 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:45.202161 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5516e813-8fcc-4ad3-bcb7-8ebb72df606c None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:45.202663 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8570b02-f046-4647-b25f-03dec0c82266 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:45.202929 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5516e813-8fcc-4ad3-bcb7-8ebb72df606c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:45.203247 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5516e813-8fcc-4ad3-bcb7-8ebb72df606c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:45.204150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8570b02-f046-4647-b25f-03dec0c82266 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:45.204683 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5516e813-8fcc-4ad3-bcb7-8ebb72df606c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:45.214711 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 56/111] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:12:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:45.220612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6e337-97ff-488b-b908-60d74340ebe5 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:45.220916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6e337-97ff-488b-b908-60d74340ebe5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:45.221201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6e337-97ff-488b-b908-60d74340ebe5 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:45.226886 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 57/112] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:45.226886 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6e337-97ff-488b-b908-60d74340ebe5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:45.294841 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:45.311768 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 56/113] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 30 16:12:45 2022] POST /identity/v3/projects => generated 302 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:12:45.314705 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5758979-7f59-43c5-a052-75522aa21214 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:45.314705 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5758979-7f59-43c5-a052-75522aa21214 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:45.314958 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5758979-7f59-43c5-a052-75522aa21214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:45.344783 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a5758979-7f59-43c5-a052-75522aa21214 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:45.346353 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a5758979-7f59-43c5-a052-75522aa21214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:45.392891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6e337-97ff-488b-b908-60d74340ebe5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:45.483551 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de7dc3-b364-41c5-b6cd-e8b4c27be0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:45.484746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de7dc3-b364-41c5-b6cd-e8b4c27be0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:45.486078 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de7dc3-b364-41c5-b6cd-e8b4c27be0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:45.487200 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de7dc3-b364-41c5-b6cd-e8b4c27be0e5 None admin] PATH_INFO: `/v3/users/2f71ac02e31149d6a110341ff612d852` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:45.506126 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de7dc3-b364-41c5-b6cd-e8b4c27be0e5 None admin] RBAC: Authorizing `identity:get_user(user_id=2f71ac02e31149d6a110341ff612d852)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:45.507921 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 57/114] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:12:45.508515 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de7dc3-b364-41c5-b6cd-e8b4c27be0e5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:45.515509 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 58/115] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:12:45 2022] GET /identity/v3/users/2f71ac02e31149d6a110341ff612d852 => generated 289 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:45.520709 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5758979-7f59-43c5-a052-75522aa21214 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:45.535554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83950c-1821-4340-bff1-0b05f5590c2c None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:45.536116 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83950c-1821-4340-bff1-0b05f5590c2c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:45.536557 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83950c-1821-4340-bff1-0b05f5590c2c None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:45.538179 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 59/116] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:45.544452 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71321430-a882-467f-95f0-8457650773ee None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:45.545350 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71321430-a882-467f-95f0-8457650773ee None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:45.545844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71321430-a882-467f-95f0-8457650773ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:45.566560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-71321430-a882-467f-95f0-8457650773ee None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:45.567812 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-71321430-a882-467f-95f0-8457650773ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:45.606074 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:45.653244 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5758979-7f59-43c5-a052-75522aa21214 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:45.659775 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 60/117] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:12:45.661396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71321430-a882-467f-95f0-8457650773ee None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:45.702435 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62caa27-d4cb-4031-925e-e71b96b7f22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:45.703729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62caa27-d4cb-4031-925e-e71b96b7f22d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:45.704200 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62caa27-d4cb-4031-925e-e71b96b7f22d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:45.704655 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62caa27-d4cb-4031-925e-e71b96b7f22d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:45.705386 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:45.790440 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62caa27-d4cb-4031-925e-e71b96b7f22d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:45.792132 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62caa27-d4cb-4031-925e-e71b96b7f22d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:45.801512 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71321430-a882-467f-95f0-8457650773ee None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:45.849767 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1259dc-5210-4864-a0ad-d26d34cc1924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:45.851312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1259dc-5210-4864-a0ad-d26d34cc1924 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:45.852087 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1259dc-5210-4864-a0ad-d26d34cc1924 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:45.853070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1259dc-5210-4864-a0ad-d26d34cc1924 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:45.859281 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:45.869770 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1259dc-5210-4864-a0ad-d26d34cc1924 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:12:45.881351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ae1259dc-5210-4864-a0ad-d26d34cc1924 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:12:45.881351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:12:45.881351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:12:45.881351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:12:45.881351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:12:45.881351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:12:45.881351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:12:45.881351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:12:45.881351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:12:45.881351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:12:45.881351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:12:45.881351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:12:45.881351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:12:45.881351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:12:45.881351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:12:45.881351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:12:45.882309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:12:45.882309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:12:45.882309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:12:45.882309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:12:45.882309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:12:45.882309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:12:45.882309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:12:45.882309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:12:45.882309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:12:45.882309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:12:45.882309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:12:45.882309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:12:45.882309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:12:45.882309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:12:45.882309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:12:45.882309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:12:45.883159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:12:45.883159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:12:45.883159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:12:45.883159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:12:45.883159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:12:45.883159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:12:45.883159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:12:45.883159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:12:45.883159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:12:45.883159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:12:45.883159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:12:45.883159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:12:45.886180 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 61/118] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 30 16:12:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:12:45.892028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52ae7b-ad75-4659-8240-f8a227697fcc None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:45.892323 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52ae7b-ad75-4659-8240-f8a227697fcc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:45.892611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52ae7b-ad75-4659-8240-f8a227697fcc None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:45.893962 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 62/119] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:45.894685 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52ae7b-ad75-4659-8240-f8a227697fcc None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:45.920338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 58/120] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:12:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:12:45.921360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62caa27-d4cb-4031-925e-e71b96b7f22d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:46.029833 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:46.032299 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62caa27-d4cb-4031-925e-e71b96b7f22d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:46.039209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52ae7b-ad75-4659-8240-f8a227697fcc None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:46.081388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7d1ee-3ba0-40c9-9cc2-f8cacf74c92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:46.082456 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c66513-8e94-4eac-947d-ac2d4a7087f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:46.082979 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7d1ee-3ba0-40c9-9cc2-f8cacf74c92a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:46.083158 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c66513-8e94-4eac-947d-ac2d4a7087f9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:46.083421 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c66513-8e94-4eac-947d-ac2d4a7087f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:46.083750 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c66513-8e94-4eac-947d-ac2d4a7087f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:46.084080 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7d1ee-3ba0-40c9-9cc2-f8cacf74c92a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:46.084554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7d1ee-3ba0-40c9-9cc2-f8cacf74c92a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:46.087364 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c66513-8e94-4eac-947d-ac2d4a7087f9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:46.087758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c66513-8e94-4eac-947d-ac2d4a7087f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:46.089191 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c66513-8e94-4eac-947d-ac2d4a7087f9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:46.098044 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7d1ee-3ba0-40c9-9cc2-f8cacf74c92a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:12:46.098206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 59/121] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:12:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:46.099402 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5be24-21e2-4706-8b78-28f186c5f8ac None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:46.104848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5be24-21e2-4706-8b78-28f186c5f8ac None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:46.105173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5be24-21e2-4706-8b78-28f186c5f8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:46.110054 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8ca7d1ee-3ba0-40c9-9cc2-f8cacf74c92a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:12:46.110054 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:12:46.110054 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:12:46.110054 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:12:46.110054 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:12:46.110054 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:12:46.110054 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:12:46.110054 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:12:46.110054 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:12:46.110054 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:12:46.110054 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:12:46.110054 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:12:46.110054 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:12:46.110054 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:12:46.110054 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:12:46.110054 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:12:46.111201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:12:46.111201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:12:46.111201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:12:46.111201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:12:46.111201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:12:46.111201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:12:46.111201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:12:46.111201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:12:46.111201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:12:46.111201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:12:46.111201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:12:46.111201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:12:46.111201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:12:46.111201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:12:46.111201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:12:46.111201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:12:46.112155 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:12:46.112155 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:12:46.112155 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:12:46.112155 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:12:46.112155 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:12:46.112155 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:12:46.112155 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:12:46.112155 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:12:46.112155 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:12:46.112155 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:12:46.112155 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:12:46.112155 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:12:46.114926 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 63/122] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:12:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:12:46.116351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7d1ee-3ba0-40c9-9cc2-f8cacf74c92a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:46.123344 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c2a5be24-21e2-4706-8b78-28f186c5f8ac None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:46.124646 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c2a5be24-21e2-4706-8b78-28f186c5f8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:46.146216 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:46.196421 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 60/123] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:12:46.197230 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5be24-21e2-4706-8b78-28f186c5f8ac None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:46.235954 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7d1ee-3ba0-40c9-9cc2-f8cacf74c92a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:46.274766 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e8244-31a2-4e67-a51e-e106a7b6ac5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:46.276508 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e8244-31a2-4e67-a51e-e106a7b6ac5f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:46.276998 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e8244-31a2-4e67-a51e-e106a7b6ac5f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:46.277532 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e8244-31a2-4e67-a51e-e106a7b6ac5f None admin] PATH_INFO: `/v3/projects/e54d8744be3c4672b24fa2e19f024af0` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:46.279085 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e8244-31a2-4e67-a51e-e106a7b6ac5f None admin] RBAC: Authorizing `identity:get_project(project_id=e54d8744be3c4672b24fa2e19f024af0)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:46.280927 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e8244-31a2-4e67-a51e-e106a7b6ac5f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:46.282967 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 64/124] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:12:46 2022] GET /identity/v3/projects/e54d8744be3c4672b24fa2e19f024af0 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:46.284310 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e8244-31a2-4e67-a51e-e106a7b6ac5f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:46.300676 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5be24-21e2-4706-8b78-28f186c5f8ac None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:46.313872 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:46.337516 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08203601-868d-4cef-979c-16abc318108d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:46.338513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08203601-868d-4cef-979c-16abc318108d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:46.338797 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08203601-868d-4cef-979c-16abc318108d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:46.339074 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08203601-868d-4cef-979c-16abc318108d None admin] PATH_INFO: `/v3/users/3181dca028754df5a6df1154d3635ddd` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:46.340314 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08203601-868d-4cef-979c-16abc318108d None admin] RBAC: Authorizing `identity:get_user(user_id=3181dca028754df5a6df1154d3635ddd)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:46.341767 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08203601-868d-4cef-979c-16abc318108d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:46.343412 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 61/125] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:12:46 2022] GET /identity/v3/users/3181dca028754df5a6df1154d3635ddd => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:46.348298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08203601-868d-4cef-979c-16abc318108d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:46.374702 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:46.404381 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e8244-31a2-4e67-a51e-e106a7b6ac5f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:46.445323 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c102c7d-1c4a-47fb-9fb7-f5fa8a0c251c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:46.446441 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c102c7d-1c4a-47fb-9fb7-f5fa8a0c251c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:46.446984 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c102c7d-1c4a-47fb-9fb7-f5fa8a0c251c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:46.447659 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c102c7d-1c4a-47fb-9fb7-f5fa8a0c251c None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:46.448922 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c102c7d-1c4a-47fb-9fb7-f5fa8a0c251c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:46.449461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c102c7d-1c4a-47fb-9fb7-f5fa8a0c251c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:46.451735 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c102c7d-1c4a-47fb-9fb7-f5fa8a0c251c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:46.462791 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 65/126] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:12:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:46.464110 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c102c7d-1c4a-47fb-9fb7-f5fa8a0c251c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:46.464341 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08203601-868d-4cef-979c-16abc318108d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:46.570142 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19ab7a-41d7-4c88-9a5c-d30de5c2f9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:46.571029 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19ab7a-41d7-4c88-9a5c-d30de5c2f9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:46.571686 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19ab7a-41d7-4c88-9a5c-d30de5c2f9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:46.572155 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f19ab7a-41d7-4c88-9a5c-d30de5c2f9f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:46.584582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f19ab7a-41d7-4c88-9a5c-d30de5c2f9f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:12:46.593326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-3f19ab7a-41d7-4c88-9a5c-d30de5c2f9f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:12:46.593326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:12:46.593326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:12:46.593326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:12:46.593326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:12:46.593326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:12:46.593326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:12:46.593326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:12:46.593326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:12:46.593326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:12:46.593326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:12:46.593326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:12:46.593326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:12:46.593326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:12:46.593326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:12:46.593326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:12:46.594221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:12:46.594221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:12:46.594221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:12:46.594221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:12:46.594221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:12:46.594221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:12:46.594221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:12:46.594221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:12:46.594221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:12:46.594221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:12:46.594221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:12:46.594221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:12:46.594221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:12:46.594221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:12:46.594221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:12:46.594221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:12:46.594973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:12:46.594973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:12:46.594973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:12:46.594973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:12:46.594973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:12:46.594973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:12:46.594973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:12:46.594973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:12:46.594973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:12:46.594973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:12:46.594973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:12:46.594973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:12:46.594973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 62/127] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:12:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:12:46.595637 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19ab7a-41d7-4c88-9a5c-d30de5c2f9f0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:46.638007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:46.655893 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c102c7d-1c4a-47fb-9fb7-f5fa8a0c251c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:46.690464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:46.693374 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abcd4a2-856a-488c-bbcf-e416faece982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:46.694706 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abcd4a2-856a-488c-bbcf-e416faece982 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:46.695194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abcd4a2-856a-488c-bbcf-e416faece982 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:46.695711 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abcd4a2-856a-488c-bbcf-e416faece982 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:46.697417 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abcd4a2-856a-488c-bbcf-e416faece982 None admin] RBAC: Adding query filter params (role.id=6e5a95d194ca42cab7a30baff4554fd3, scope.project.id=e54d8744be3c4672b24fa2e19f024af0, user.id=2f71ac02e31149d6a110341ff612d852) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:46.697994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abcd4a2-856a-488c-bbcf-e416faece982 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:46.699843 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abcd4a2-856a-488c-bbcf-e416faece982 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:46.706195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 66/128] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:12:46 2022] GET /identity/v3/role_assignments?user.id=2f71ac02e31149d6a110341ff612d852&scope.project.id=e54d8744be3c4672b24fa2e19f024af0&role.id=6e5a95d194ca42cab7a30baff4554fd3 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:46.706773 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f19ab7a-41d7-4c88-9a5c-d30de5c2f9f0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:46.707662 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abcd4a2-856a-488c-bbcf-e416faece982 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:46.744702 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60677451-c908-4de7-88f2-54dcb0b18fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:46.745571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60677451-c908-4de7-88f2-54dcb0b18fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:46.745858 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60677451-c908-4de7-88f2-54dcb0b18fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:46.746140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60677451-c908-4de7-88f2-54dcb0b18fb1 None admin] PATH_INFO: `/v3/projects/e54d8744be3c4672b24fa2e19f024af0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:46.747446 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60677451-c908-4de7-88f2-54dcb0b18fb1 None admin] RBAC: Authorizing `identity:get_project(project_id=e54d8744be3c4672b24fa2e19f024af0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:46.749024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60677451-c908-4de7-88f2-54dcb0b18fb1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:46.751197 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 63/129] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:12:46 2022] GET /identity/v3/projects/e54d8744be3c4672b24fa2e19f024af0 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:46.751718 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60677451-c908-4de7-88f2-54dcb0b18fb1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:46.827632 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abcd4a2-856a-488c-bbcf-e416faece982 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:46.869214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b8d1c-5786-4eab-9ad4-0f934ce9fcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:46.870708 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b8d1c-5786-4eab-9ad4-0f934ce9fcf8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:46.872022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b8d1c-5786-4eab-9ad4-0f934ce9fcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:46.872688 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028b8d1c-5786-4eab-9ad4-0f934ce9fcf8 None admin] PATH_INFO: `/v3/users/9242c0a59f6b49568f911aaa0ef688f2` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:46.875799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b8d1c-5786-4eab-9ad4-0f934ce9fcf8 None admin] RBAC: Authorizing `identity:get_user(user_id=9242c0a59f6b49568f911aaa0ef688f2)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:46.879150 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028b8d1c-5786-4eab-9ad4-0f934ce9fcf8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:46.881347 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 67/130] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:12:46 2022] GET /identity/v3/users/9242c0a59f6b49568f911aaa0ef688f2 => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:46.883625 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60677451-c908-4de7-88f2-54dcb0b18fb1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:46.887671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b8d1c-5786-4eab-9ad4-0f934ce9fcf8 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:46.928126 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:46.964646 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817c162-c437-4351-9cfd-9aadfd931f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:46.965331 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817c162-c437-4351-9cfd-9aadfd931f0d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:46.965617 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817c162-c437-4351-9cfd-9aadfd931f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:46.965895 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817c162-c437-4351-9cfd-9aadfd931f0d None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:46.966660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817c162-c437-4351-9cfd-9aadfd931f0d None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:46.966971 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817c162-c437-4351-9cfd-9aadfd931f0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:46.969779 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817c162-c437-4351-9cfd-9aadfd931f0d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:46.980749 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 64/131] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:12:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:46.981602 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817c162-c437-4351-9cfd-9aadfd931f0d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:47.003415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:47.047134 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028b8d1c-5786-4eab-9ad4-0f934ce9fcf8 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:47.097295 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817c162-c437-4351-9cfd-9aadfd931f0d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:47.104224 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5b1ac-381b-4e29-8a63-82bed0836b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:47.105146 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5b1ac-381b-4e29-8a63-82bed0836b17 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:47.105436 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5b1ac-381b-4e29-8a63-82bed0836b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:47.105758 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5b1ac-381b-4e29-8a63-82bed0836b17 None admin] PATH_INFO: `/v3/projects/e54d8744be3c4672b24fa2e19f024af0/users/3181dca028754df5a6df1154d3635ddd/roles/cfeec012f4ec48138c0e637e9a7fde61` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:47.112343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5b1ac-381b-4e29-8a63-82bed0836b17 None admin] RBAC: Authorizing `identity:create_grant(project_id=e54d8744be3c4672b24fa2e19f024af0, user_id=3181dca028754df5a6df1154d3635ddd, role_id=cfeec012f4ec48138c0e637e9a7fde61)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:47.113902 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5b1ac-381b-4e29-8a63-82bed0836b17 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:47.138182 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3561a2-4479-4687-82e4-cb55a4dac6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:47.139424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3561a2-4479-4687-82e4-cb55a4dac6e5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:47.139961 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3561a2-4479-4687-82e4-cb55a4dac6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:47.140458 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3561a2-4479-4687-82e4-cb55a4dac6e5 None admin] PATH_INFO: `/v3/projects/e54d8744be3c4672b24fa2e19f024af0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:47.141992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3561a2-4479-4687-82e4-cb55a4dac6e5 None admin] RBAC: Authorizing `identity:get_project(project_id=e54d8744be3c4672b24fa2e19f024af0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:47.143720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3561a2-4479-4687-82e4-cb55a4dac6e5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:47.145670 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 65/132] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:12:46 2022] GET /identity/v3/projects/e54d8744be3c4672b24fa2e19f024af0 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:47.146626 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3561a2-4479-4687-82e4-cb55a4dac6e5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:47.239110 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:47.259070 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3561a2-4479-4687-82e4-cb55a4dac6e5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:47.265831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 68/133] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 30 16:12:46 2022] PUT /identity/v3/projects/e54d8744be3c4672b24fa2e19f024af0/users/3181dca028754df5a6df1154d3635ddd/roles/cfeec012f4ec48138c0e637e9a7fde61 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:12:47.266752 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5b1ac-381b-4e29-8a63-82bed0836b17 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:47.307345 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52c804-0bf4-488e-939c-c558a734fc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:47.308688 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52c804-0bf4-488e-939c-c558a734fc0c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:47.309201 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52c804-0bf4-488e-939c-c558a734fc0c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:47.309923 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52c804-0bf4-488e-939c-c558a734fc0c None admin] PATH_INFO: `/v3/users/3181dca028754df5a6df1154d3635ddd` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:47.314488 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f52c804-0bf4-488e-939c-c558a734fc0c None admin] RBAC: Authorizing `identity:get_user(user_id=3181dca028754df5a6df1154d3635ddd)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:47.316198 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f52c804-0bf4-488e-939c-c558a734fc0c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:47.318092 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 66/134] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:12:47 2022] GET /identity/v3/users/3181dca028754df5a6df1154d3635ddd => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:47.331601 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52c804-0bf4-488e-939c-c558a734fc0c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:47.431931 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:47.502028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5b1ac-381b-4e29-8a63-82bed0836b17 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:47.569659 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbe86f-a84e-48ab-a1a3-d82dc7c0e4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:47.569881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52c804-0bf4-488e-939c-c558a734fc0c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:47.571044 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbe86f-a84e-48ab-a1a3-d82dc7c0e4e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:47.572200 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbe86f-a84e-48ab-a1a3-d82dc7c0e4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:47.572685 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbe86f-a84e-48ab-a1a3-d82dc7c0e4e9 None admin] PATH_INFO: `/v3/projects/e54d8744be3c4672b24fa2e19f024af0/users/9242c0a59f6b49568f911aaa0ef688f2/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:47.574969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cbe86f-a84e-48ab-a1a3-d82dc7c0e4e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=e54d8744be3c4672b24fa2e19f024af0, user_id=9242c0a59f6b49568f911aaa0ef688f2, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:47.577712 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cbe86f-a84e-48ab-a1a3-d82dc7c0e4e9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:47.627489 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5b8599-98f3-4e7d-8015-390a46e9288a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:47.628768 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b8599-98f3-4e7d-8015-390a46e9288a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:47.629248 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b8599-98f3-4e7d-8015-390a46e9288a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:47.629740 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5b8599-98f3-4e7d-8015-390a46e9288a None admin] PATH_INFO: `/v3/projects/54d37e8eb5424296b3a4c9602e06061d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:47.637084 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b8599-98f3-4e7d-8015-390a46e9288a None admin] RBAC: Authorizing `identity:get_project(project_id=54d37e8eb5424296b3a4c9602e06061d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:47.640019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 69/135] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 30 16:12:47 2022] PUT /identity/v3/projects/e54d8744be3c4672b24fa2e19f024af0/users/9242c0a59f6b49568f911aaa0ef688f2/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:12:47.642062 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5b8599-98f3-4e7d-8015-390a46e9288a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:47.644342 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 67/136] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:12:47 2022] GET /identity/v3/projects/54d37e8eb5424296b3a4c9602e06061d => generated 312 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:12:47.651380 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbe86f-a84e-48ab-a1a3-d82dc7c0e4e9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:47.728627 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c1f96-c709-4157-ab66-2e300926e0fc None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:47.729253 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c1f96-c709-4157-ab66-2e300926e0fc None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:47.729733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c1f96-c709-4157-ab66-2e300926e0fc None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:47.731458 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 68/137] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:12:47.736976 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052177f-7514-4e3e-a4be-3ac01257bd4d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:47.737580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052177f-7514-4e3e-a4be-3ac01257bd4d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:47.738063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052177f-7514-4e3e-a4be-3ac01257bd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:47.773002 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6052177f-7514-4e3e-a4be-3ac01257bd4d None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:47.775280 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6052177f-7514-4e3e-a4be-3ac01257bd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:47.796422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:47.804347 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbe86f-a84e-48ab-a1a3-d82dc7c0e4e9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:47.874433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dffbb14-82bb-40c3-b219-e7924a1e1cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:47.875669 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffbb14-82bb-40c3-b219-e7924a1e1cc1 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:47.876133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffbb14-82bb-40c3-b219-e7924a1e1cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:47.876584 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffbb14-82bb-40c3-b219-e7924a1e1cc1 None admin] PATH_INFO: `/v3/projects/54d37e8eb5424296b3a4c9602e06061d/users/3181dca028754df5a6df1154d3635ddd/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:47.878857 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dffbb14-82bb-40c3-b219-e7924a1e1cc1 None admin] RBAC: Authorizing `identity:create_grant(project_id=54d37e8eb5424296b3a4c9602e06061d, user_id=3181dca028754df5a6df1154d3635ddd, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:47.881283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dffbb14-82bb-40c3-b219-e7924a1e1cc1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:47.887249 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:47.888998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 69/138] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:12:47.889714 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06269c85-0c42-489a-bb35-128f70d32d19 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:47.890012 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06269c85-0c42-489a-bb35-128f70d32d19 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:47.890367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06269c85-0c42-489a-bb35-128f70d32d19 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:47.891751 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 70/139] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:47.897060 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e9cdc-1e80-42af-8b38-0e23f7ccc71f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:47.897354 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e9cdc-1e80-42af-8b38-0e23f7ccc71f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:47.897636 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e9cdc-1e80-42af-8b38-0e23f7ccc71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:47.922015 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0d1e9cdc-1e80-42af-8b38-0e23f7ccc71f None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:47.923659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0d1e9cdc-1e80-42af-8b38-0e23f7ccc71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:47.975877 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 70/140] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 30 16:12:47 2022] PUT /identity/v3/projects/54d37e8eb5424296b3a4c9602e06061d/users/3181dca028754df5a6df1154d3635ddd/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:12:47.977079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dffbb14-82bb-40c3-b219-e7924a1e1cc1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:48.059928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 71/141] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:12:48.096514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e9cdc-1e80-42af-8b38-0e23f7ccc71f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:48.135801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dffbb14-82bb-40c3-b219-e7924a1e1cc1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:48.194746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23cebd9-6e8e-4d32-abda-346d359dd6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:48.195916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23cebd9-6e8e-4d32-abda-346d359dd6ea None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:48.196458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23cebd9-6e8e-4d32-abda-346d359dd6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:48.196952 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23cebd9-6e8e-4d32-abda-346d359dd6ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:48.198466 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23cebd9-6e8e-4d32-abda-346d359dd6ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:48.200296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23cebd9-6e8e-4d32-abda-346d359dd6ea None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:48.202652 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 71/142] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:12:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:48.208586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23cebd9-6e8e-4d32-abda-346d359dd6ea None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:48.280334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:48.321617 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e9cdc-1e80-42af-8b38-0e23f7ccc71f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:48.326613 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:48.372812 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23cebd9-6e8e-4d32-abda-346d359dd6ea None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:48.422474 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352616a-d5cc-4e55-8235-f7d79913af7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:48.423172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352616a-d5cc-4e55-8235-f7d79913af7c None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:48.423459 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352616a-d5cc-4e55-8235-f7d79913af7c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:48.423783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352616a-d5cc-4e55-8235-f7d79913af7c None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:48.424578 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352616a-d5cc-4e55-8235-f7d79913af7c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:48.426410 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352616a-d5cc-4e55-8235-f7d79913af7c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:48.429091 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418a3ac-ce25-4bac-aaa0-7885be6f199e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:48.429909 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418a3ac-ce25-4bac-aaa0-7885be6f199e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:48.430193 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418a3ac-ce25-4bac-aaa0-7885be6f199e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:48.430468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418a3ac-ce25-4bac-aaa0-7885be6f199e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:48.464447 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b418a3ac-ce25-4bac-aaa0-7885be6f199e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:12:48.472387 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-b418a3ac-ce25-4bac-aaa0-7885be6f199e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:12:48.472387 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:12:48.472387 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:12:48.472387 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:12:48.472387 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:12:48.472387 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:12:48.472387 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:12:48.472387 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:12:48.472387 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:12:48.472387 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:12:48.472387 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:12:48.472387 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:12:48.472387 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:12:48.472387 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:12:48.472387 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:12:48.472387 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:12:48.473247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:12:48.473247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:12:48.473247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:12:48.473247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:12:48.473247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:12:48.473247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:12:48.473247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:12:48.473247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:12:48.473247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:12:48.473247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:12:48.473247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:12:48.473247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:12:48.473247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:12:48.473247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:12:48.473247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:12:48.473247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:12:48.474024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:12:48.474024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:12:48.474024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:12:48.474024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:12:48.474024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:12:48.474024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:12:48.474024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:12:48.474024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:12:48.474024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:12:48.474024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:12:48.474024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:12:48.474024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:12:48.478849 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 72/143] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:12:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:12:48.479908 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418a3ac-ce25-4bac-aaa0-7885be6f199e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:48.546222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 72/144] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 30 16:12:48 2022] POST /identity/v3/users => generated 283 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:12:48.624372 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418a3ac-ce25-4bac-aaa0-7885be6f199e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:48.672140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7a3ac-0863-4433-94da-0c6687debe74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:48.672829 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7a3ac-0863-4433-94da-0c6687debe74 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:48.673113 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7a3ac-0863-4433-94da-0c6687debe74 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:48.673386 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7a3ac-0863-4433-94da-0c6687debe74 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:48.674208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af7a3ac-0863-4433-94da-0c6687debe74 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:48.674515 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af7a3ac-0863-4433-94da-0c6687debe74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:48.676010 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af7a3ac-0863-4433-94da-0c6687debe74 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:48.688342 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:48.692209 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 73/145] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:12:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:12:48.697944 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352616a-d5cc-4e55-8235-f7d79913af7c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:48.844398 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352616a-d5cc-4e55-8235-f7d79913af7c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:48.875856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:48.895011 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b94c7-54e8-4363-99b5-febe23ec64a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:48.896001 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b94c7-54e8-4363-99b5-febe23ec64a1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:48.896296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b94c7-54e8-4363-99b5-febe23ec64a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:48.896671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b94c7-54e8-4363-99b5-febe23ec64a1 None admin] PATH_INFO: `/v3/users/3181dca028754df5a6df1154d3635ddd` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:48.897975 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b94c7-54e8-4363-99b5-febe23ec64a1 None admin] RBAC: Authorizing `identity:get_user(user_id=3181dca028754df5a6df1154d3635ddd)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:48.900143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b94c7-54e8-4363-99b5-febe23ec64a1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:48.902359 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 73/146] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:12:48 2022] GET /identity/v3/users/3181dca028754df5a6df1154d3635ddd => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:48.908423 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7a3ac-0863-4433-94da-0c6687debe74 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:49.097494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7a3ac-0863-4433-94da-0c6687debe74 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:49.146539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138386ca-17e7-4aff-9939-a1d7d3f6683a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:49.147201 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138386ca-17e7-4aff-9939-a1d7d3f6683a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:49.147509 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138386ca-17e7-4aff-9939-a1d7d3f6683a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:49.147791 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138386ca-17e7-4aff-9939-a1d7d3f6683a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:49.232753 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:49.235030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138386ca-17e7-4aff-9939-a1d7d3f6683a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:49.236810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138386ca-17e7-4aff-9939-a1d7d3f6683a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:49.354815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 74/147] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:12:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:12:49.361129 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b94c7-54e8-4363-99b5-febe23ec64a1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:49.513627 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b94c7-54e8-4363-99b5-febe23ec64a1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:49.562040 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24833ffa-ca14-4bc8-8559-a6219956b4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:49.563009 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24833ffa-ca14-4bc8-8559-a6219956b4df None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:49.563294 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24833ffa-ca14-4bc8-8559-a6219956b4df None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:49.563592 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24833ffa-ca14-4bc8-8559-a6219956b4df None admin] PATH_INFO: `/v3/projects/e54d8744be3c4672b24fa2e19f024af0` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:49.568218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24833ffa-ca14-4bc8-8559-a6219956b4df None admin] RBAC: Authorizing `identity:get_project(project_id=e54d8744be3c4672b24fa2e19f024af0)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:49.569722 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24833ffa-ca14-4bc8-8559-a6219956b4df None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:49.572150 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 74/148] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:12:49 2022] GET /identity/v3/projects/e54d8744be3c4672b24fa2e19f024af0 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:49.577452 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138386ca-17e7-4aff-9939-a1d7d3f6683a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:49.647993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d459bc-def7-41ea-be6c-f7153b4dd7e2 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:49.648304 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d459bc-def7-41ea-be6c-f7153b4dd7e2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:49.648589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d459bc-def7-41ea-be6c-f7153b4dd7e2 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:49.650905 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 75/149] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:12:49.656355 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0300a0-7943-4052-bb10-524c1393cbce None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:49.656646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0300a0-7943-4052-bb10-524c1393cbce None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:49.656929 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0300a0-7943-4052-bb10-524c1393cbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:49.692511 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9d0300a0-7943-4052-bb10-524c1393cbce None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:49.693734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9d0300a0-7943-4052-bb10-524c1393cbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:49.747331 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:49.751857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138386ca-17e7-4aff-9939-a1d7d3f6683a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:49.821019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d832b-4a1a-424a-84e7-0322471e74d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:49.822513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d832b-4a1a-424a-84e7-0322471e74d8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:49.822757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d832b-4a1a-424a-84e7-0322471e74d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:49.823033 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d832b-4a1a-424a-84e7-0322471e74d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:49.824112 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d832b-4a1a-424a-84e7-0322471e74d8 None admin] RBAC: Adding query filter params (role.id=9ab3e302ead54950b7c30103f1e7fda7, scope.project.id=e54d8744be3c4672b24fa2e19f024af0, user.id=3181dca028754df5a6df1154d3635ddd) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:49.824435 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d832b-4a1a-424a-84e7-0322471e74d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:49.826138 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d832b-4a1a-424a-84e7-0322471e74d8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:49.842016 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 75/150] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:12:49 2022] GET /identity/v3/role_assignments?user.id=3181dca028754df5a6df1154d3635ddd&scope.project.id=e54d8744be3c4672b24fa2e19f024af0&role.id=9ab3e302ead54950b7c30103f1e7fda7 => generated 608 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:49.851950 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:49.881240 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 76/151] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:12:49.922639 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d832b-4a1a-424a-84e7-0322471e74d8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:50.089194 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d832b-4a1a-424a-84e7-0322471e74d8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:50.140559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9a43ae-75e7-435b-9c33-e189eaa239ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:50.141373 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a43ae-75e7-435b-9c33-e189eaa239ce None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:50.141659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a43ae-75e7-435b-9c33-e189eaa239ce None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:50.141935 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9a43ae-75e7-435b-9c33-e189eaa239ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:50.158164 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9a43ae-75e7-435b-9c33-e189eaa239ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:12:50.166659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-0c9a43ae-75e7-435b-9c33-e189eaa239ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 16:12:50.166659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:12:50.166659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:12:50.166659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:12:50.166659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:12:50.166659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:12:50.166659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:12:50.166659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:12:50.166659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:12:50.166659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:12:50.166659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:12:50.166659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:12:50.166659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:12:50.166659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:12:50.166659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:12:50.166659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:12:50.167603 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:12:50.167603 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:12:50.167603 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:12:50.167603 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:12:50.167603 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:12:50.167603 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:12:50.167603 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:12:50.167603 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:12:50.167603 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:12:50.167603 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:12:50.167603 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:12:50.167603 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:12:50.167603 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:12:50.167603 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:12:50.167603 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:12:50.167603 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:12:50.168381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:12:50.168381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:12:50.168381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:12:50.168381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:12:50.168381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:12:50.168381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:12:50.168381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:12:50.168381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:12:50.168381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:12:50.168381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:12:50.168381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 16:12:50.168381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:12:50.168381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 76/152] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:12:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:12:50.175860 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0300a0-7943-4052-bb10-524c1393cbce None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:50.307228 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325dab0d-a886-4964-b4e4-ad484d391c83 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:50.307590 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325dab0d-a886-4964-b4e4-ad484d391c83 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:50.307849 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325dab0d-a886-4964-b4e4-ad484d391c83 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:50.309234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 77/153] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:12:50.314727 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2edf4a-95d5-4703-ba91-698fbe54fa89 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:50.315017 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2edf4a-95d5-4703-ba91-698fbe54fa89 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:50.315299 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2edf4a-95d5-4703-ba91-698fbe54fa89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:50.332240 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0300a0-7943-4052-bb10-524c1393cbce None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:50.344710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6e2edf4a-95d5-4703-ba91-698fbe54fa89 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:50.346340 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6e2edf4a-95d5-4703-ba91-698fbe54fa89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:50.375828 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:50.379454 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0436d1b-c437-46bf-9e23-1bb493904e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:50.380225 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0436d1b-c437-46bf-9e23-1bb493904e7c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:50.380512 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0436d1b-c437-46bf-9e23-1bb493904e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:50.380793 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0436d1b-c437-46bf-9e23-1bb493904e7c None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:50.381561 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0436d1b-c437-46bf-9e23-1bb493904e7c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:50.381876 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0436d1b-c437-46bf-9e23-1bb493904e7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:50.383284 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0436d1b-c437-46bf-9e23-1bb493904e7c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:50.395394 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 77/154] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:12:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:50.401874 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0436d1b-c437-46bf-9e23-1bb493904e7c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:50.458228 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 78/155] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:12:50.487212 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:50.510694 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2edf4a-95d5-4703-ba91-698fbe54fa89 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:50.553220 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0436d1b-c437-46bf-9e23-1bb493904e7c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:50.608819 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb63be-a4de-451a-9483-86c9656d0817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:50.610138 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb63be-a4de-451a-9483-86c9656d0817 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:50.610626 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb63be-a4de-451a-9483-86c9656d0817 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:50.611101 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb63be-a4de-451a-9483-86c9656d0817 None admin] PATH_INFO: `/v3/users/2f71ac02e31149d6a110341ff612d852` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:50.612770 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb63be-a4de-451a-9483-86c9656d0817 None admin] RBAC: Authorizing `identity:get_user(user_id=2f71ac02e31149d6a110341ff612d852)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:50.614549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb63be-a4de-451a-9483-86c9656d0817 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:50.620709 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 78/156] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:12:50 2022] GET /identity/v3/users/2f71ac02e31149d6a110341ff612d852 => generated 289 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:12:50.626485 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb63be-a4de-451a-9483-86c9656d0817 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:50.654999 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2edf4a-95d5-4703-ba91-698fbe54fa89 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:50.716486 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd7830-f069-4abc-864b-664aa5dbb9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:50.717241 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd7830-f069-4abc-864b-664aa5dbb9c7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:50.717572 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd7830-f069-4abc-864b-664aa5dbb9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:50.717791 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bd7830-f069-4abc-864b-664aa5dbb9c7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:50.745720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bd7830-f069-4abc-864b-664aa5dbb9c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:12:50.753990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a7bd7830-f069-4abc-864b-664aa5dbb9c7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:12:50.753990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:12:50.753990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:12:50.753990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:12:50.753990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:12:50.753990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:12:50.753990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:12:50.753990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:12:50.753990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:12:50.753990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:12:50.753990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:12:50.753990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:12:50.753990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:12:50.753990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:12:50.753990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:12:50.753990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:12:50.754845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:12:50.754845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:12:50.754845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:12:50.754845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:12:50.754845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:12:50.754845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:12:50.754845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:12:50.754845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:12:50.754845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:12:50.754845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:12:50.754845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:12:50.754845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:12:50.754845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:12:50.754845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:12:50.754845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:12:50.754845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:12:50.755640 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:12:50.755640 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:12:50.755640 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:12:50.755640 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:12:50.755640 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:12:50.755640 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:12:50.755640 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:12:50.755640 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:12:50.755640 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:12:50.755640 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:12:50.755640 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:12:50.755640 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:12:50.755640 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 79/157] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 30 16:12:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:12:50.768005 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd7830-f069-4abc-864b-664aa5dbb9c7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:50.797608 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:50.827851 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb63be-a4de-451a-9483-86c9656d0817 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:50.865167 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:50.904034 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f37cd-3ce6-4a27-a1d1-b7527ff57ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:50.904910 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f37cd-3ce6-4a27-a1d1-b7527ff57ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:50.905206 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f37cd-3ce6-4a27-a1d1-b7527ff57ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:50.905478 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f37cd-3ce6-4a27-a1d1-b7527ff57ca7 None admin] PATH_INFO: `/v3/projects/e54d8744be3c4672b24fa2e19f024af0` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:50.906783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f37cd-3ce6-4a27-a1d1-b7527ff57ca7 None admin] RBAC: Authorizing `identity:get_project(project_id=e54d8744be3c4672b24fa2e19f024af0)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:50.908343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f37cd-3ce6-4a27-a1d1-b7527ff57ca7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:50.910089 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 79/158] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:12:50 2022] GET /identity/v3/projects/e54d8744be3c4672b24fa2e19f024af0 => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:12:50.911344 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103b802-5a49-4ee7-add0-bd91693c55b4 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:50.913159 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bd7830-f069-4abc-864b-664aa5dbb9c7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:50.915736 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103b802-5a49-4ee7-add0-bd91693c55b4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:50.916263 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103b802-5a49-4ee7-add0-bd91693c55b4 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:50.917632 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 80/159] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:50.918439 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103b802-5a49-4ee7-add0-bd91693c55b4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:50.973127 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4020eb77-5d5e-4344-a29c-e886c9745044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:50.973770 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020eb77-5d5e-4344-a29c-e886c9745044 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:50.974050 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020eb77-5d5e-4344-a29c-e886c9745044 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:50.974326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4020eb77-5d5e-4344-a29c-e886c9745044 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:50.975078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4020eb77-5d5e-4344-a29c-e886c9745044 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:50.975384 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4020eb77-5d5e-4344-a29c-e886c9745044 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:50.980565 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4020eb77-5d5e-4344-a29c-e886c9745044 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:50.995680 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 80/160] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:12:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:12:50.999248 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f6f2f-6f76-4b00-b63b-4932c465cdac None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:50.999553 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f6f2f-6f76-4b00-b63b-4932c465cdac None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:50.999830 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f6f2f-6f76-4b00-b63b-4932c465cdac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:51.045620 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d95f6f2f-6f76-4b00-b63b-4932c465cdac None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:51.046834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d95f6f2f-6f76-4b00-b63b-4932c465cdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:51.120679 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103b802-5a49-4ee7-add0-bd91693c55b4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:51.193646 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 81/161] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:12:51.194601 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f6f2f-6f76-4b00-b63b-4932c465cdac None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:51.198690 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f60b5-c39b-4a83-861e-4862b678b695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:51.199914 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f60b5-c39b-4a83-861e-4862b678b695 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:51.200405 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f60b5-c39b-4a83-861e-4862b678b695 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:51.200881 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f60b5-c39b-4a83-861e-4862b678b695 None admin] PATH_INFO: `/v3/projects/e54d8744be3c4672b24fa2e19f024af0/users/2f71ac02e31149d6a110341ff612d852/roles/6e5a95d194ca42cab7a30baff4554fd3` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:51.208920 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f60b5-c39b-4a83-861e-4862b678b695 None admin] RBAC: Authorizing `identity:create_grant(project_id=e54d8744be3c4672b24fa2e19f024af0, user_id=2f71ac02e31149d6a110341ff612d852, role_id=6e5a95d194ca42cab7a30baff4554fd3)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:51.210805 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f60b5-c39b-4a83-861e-4862b678b695 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:51.212373 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:51.302281 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 81/162] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 30 16:12:50 2022] PUT /identity/v3/projects/e54d8744be3c4672b24fa2e19f024af0/users/2f71ac02e31149d6a110341ff612d852/roles/6e5a95d194ca42cab7a30baff4554fd3 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:12:51.303211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f60b5-c39b-4a83-861e-4862b678b695 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:51.373510 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f6f2f-6f76-4b00-b63b-4932c465cdac None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:51.381659 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:51.475138 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f60b5-c39b-4a83-861e-4862b678b695 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:51.484236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ed59a-634e-4150-9d36-45ad96dbfffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:51.485143 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ed59a-634e-4150-9d36-45ad96dbfffd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:51.485423 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ed59a-634e-4150-9d36-45ad96dbfffd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:51.485704 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ed59a-634e-4150-9d36-45ad96dbfffd None admin] PATH_INFO: `/v3/users/3181dca028754df5a6df1154d3635ddd` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:51.486982 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ed59a-634e-4150-9d36-45ad96dbfffd None admin] RBAC: Authorizing `identity:get_user(user_id=3181dca028754df5a6df1154d3635ddd)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:51.488467 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ed59a-634e-4150-9d36-45ad96dbfffd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:51.490129 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 82/163] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:12:51 2022] GET /identity/v3/users/3181dca028754df5a6df1154d3635ddd => generated 275 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:51.495666 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ed59a-634e-4150-9d36-45ad96dbfffd None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:51.522849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed700ef-e0e8-482e-9084-62d57f9f9cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:51.523717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed700ef-e0e8-482e-9084-62d57f9f9cbf None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:51.524005 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed700ef-e0e8-482e-9084-62d57f9f9cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:51.525841 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed700ef-e0e8-482e-9084-62d57f9f9cbf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:51.543509 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed700ef-e0e8-482e-9084-62d57f9f9cbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:12:51.551296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-5ed700ef-e0e8-482e-9084-62d57f9f9cbf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:12:51.551296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:12:51.551296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:12:51.551296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:12:51.551296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:12:51.551296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:12:51.551296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:12:51.551296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:12:51.551296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:12:51.551296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:12:51.551296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:12:51.551296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:12:51.551296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:12:51.551296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:12:51.551296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:12:51.551296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:12:51.552269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:12:51.552269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:12:51.552269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:12:51.552269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:12:51.552269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:12:51.552269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:12:51.552269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:12:51.552269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:12:51.552269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:12:51.552269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:12:51.552269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:12:51.552269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:12:51.552269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:12:51.552269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:12:51.552269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:12:51.552269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:12:51.553085 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:12:51.553085 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:12:51.553085 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:12:51.553085 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:12:51.553085 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:12:51.553085 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:12:51.553085 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:12:51.553085 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:12:51.553085 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:12:51.553085 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:12:51.553085 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:12:51.553085 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:12:51.560946 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 82/164] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:12:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:12:51.567572 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed700ef-e0e8-482e-9084-62d57f9f9cbf None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:51.683833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:51.684951 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed700ef-e0e8-482e-9084-62d57f9f9cbf None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:51.687233 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:51.727056 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17fc88f-7439-4ac9-a1dd-3f7f3fd0c601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:51.729074 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fc88f-7439-4ac9-a1dd-3f7f3fd0c601 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:51.729358 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fc88f-7439-4ac9-a1dd-3f7f3fd0c601 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:51.729643 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fc88f-7439-4ac9-a1dd-3f7f3fd0c601 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:51.730401 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17fc88f-7439-4ac9-a1dd-3f7f3fd0c601 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:51.730710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17fc88f-7439-4ac9-a1dd-3f7f3fd0c601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:51.732140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17fc88f-7439-4ac9-a1dd-3f7f3fd0c601 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:51.737382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ed59a-634e-4150-9d36-45ad96dbfffd None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:51.741029 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 83/165] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:12:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:12:51.747178 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17fc88f-7439-4ac9-a1dd-3f7f3fd0c601 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:51.782948 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7464f5-0e96-466c-a1cc-1217468262c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:51.784118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7464f5-0e96-466c-a1cc-1217468262c2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:51.784629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7464f5-0e96-466c-a1cc-1217468262c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:51.785108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7464f5-0e96-466c-a1cc-1217468262c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:51.877879 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7464f5-0e96-466c-a1cc-1217468262c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:51.879736 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7464f5-0e96-466c-a1cc-1217468262c2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:51.910243 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17fc88f-7439-4ac9-a1dd-3f7f3fd0c601 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:51.966624 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29ef3c-29b3-4dd3-a4a3-4fa7ca17b195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:51.967529 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29ef3c-29b3-4dd3-a4a3-4fa7ca17b195 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:51.967863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29ef3c-29b3-4dd3-a4a3-4fa7ca17b195 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:51.968140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29ef3c-29b3-4dd3-a4a3-4fa7ca17b195 None admin] PATH_INFO: `/v3/users/9242c0a59f6b49568f911aaa0ef688f2` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:51.972209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29ef3c-29b3-4dd3-a4a3-4fa7ca17b195 None admin] RBAC: Authorizing `identity:get_user(user_id=9242c0a59f6b49568f911aaa0ef688f2)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:51.973051 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:51.973855 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d29ef3c-29b3-4dd3-a4a3-4fa7ca17b195 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:51.975530 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 84/166] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:12:51 2022] GET /identity/v3/users/9242c0a59f6b49568f911aaa0ef688f2 => generated 247 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:51.976590 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea98915-54c6-47df-ada1-1e404ae35a00 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:51.976873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea98915-54c6-47df-ada1-1e404ae35a00 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:51.977149 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea98915-54c6-47df-ada1-1e404ae35a00 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:51.979887 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 85/167] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:51.983683 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea98915-54c6-47df-ada1-1e404ae35a00 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:52.018458 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 83/168] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:12:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:12:52.020494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49681c07-984b-4ee4-90cc-e424833e33ac None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:52.021015 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49681c07-984b-4ee4-90cc-e424833e33ac None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:52.021492 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49681c07-984b-4ee4-90cc-e424833e33ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:52.040984 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-49681c07-984b-4ee4-90cc-e424833e33ac None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:52.042560 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-49681c07-984b-4ee4-90cc-e424833e33ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:52.049239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:52.122903 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 84/169] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:12:52.124137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49681c07-984b-4ee4-90cc-e424833e33ac None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:52.161312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea98915-54c6-47df-ada1-1e404ae35a00 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:52.239136 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c268c-21b2-4ce0-8b3b-431234c7e24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:52.239604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c268c-21b2-4ce0-8b3b-431234c7e24e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:52.239889 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c268c-21b2-4ce0-8b3b-431234c7e24e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:52.240165 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c268c-21b2-4ce0-8b3b-431234c7e24e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:52.243210 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49681c07-984b-4ee4-90cc-e424833e33ac None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:52.246189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:52.283082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba662018-619f-454b-ae7f-ccec9788d93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:52.284392 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba662018-619f-454b-ae7f-ccec9788d93c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:52.284882 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba662018-619f-454b-ae7f-ccec9788d93c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:52.285372 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba662018-619f-454b-ae7f-ccec9788d93c None admin] PATH_INFO: `/v3/projects/e54d8744be3c4672b24fa2e19f024af0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:52.286773 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba662018-619f-454b-ae7f-ccec9788d93c None admin] RBAC: Authorizing `identity:get_project(project_id=e54d8744be3c4672b24fa2e19f024af0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:52.288491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba662018-619f-454b-ae7f-ccec9788d93c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:52.290611 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 85/170] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:12:52 2022] GET /identity/v3/projects/e54d8744be3c4672b24fa2e19f024af0 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:52.291735 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba662018-619f-454b-ae7f-ccec9788d93c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:52.400802 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c268c-21b2-4ce0-8b3b-431234c7e24e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:52.402367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c268c-21b2-4ce0-8b3b-431234c7e24e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:52.407624 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba662018-619f-454b-ae7f-ccec9788d93c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:52.451501 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b567bd2-771a-41c4-8e58-9dacf6bdc53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:52.452626 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b567bd2-771a-41c4-8e58-9dacf6bdc53d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:52.453119 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b567bd2-771a-41c4-8e58-9dacf6bdc53d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:52.453592 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b567bd2-771a-41c4-8e58-9dacf6bdc53d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:52.463649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:52.469693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b567bd2-771a-41c4-8e58-9dacf6bdc53d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:12:52.479571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-4b567bd2-771a-41c4-8e58-9dacf6bdc53d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:12:52.479571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:12:52.479571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:12:52.479571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:12:52.479571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:12:52.479571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:12:52.479571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:12:52.479571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:12:52.479571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:12:52.479571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:12:52.479571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:12:52.479571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:12:52.479571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:12:52.479571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:12:52.479571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:12:52.479571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:12:52.480443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:12:52.480443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:12:52.480443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:12:52.480443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:12:52.480443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:12:52.480443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:12:52.480443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:12:52.480443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:12:52.480443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:12:52.480443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:12:52.480443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:12:52.480443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:12:52.480443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:12:52.480443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:12:52.480443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:12:52.480443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:12:52.481304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:12:52.481304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:12:52.481304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:12:52.481304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:12:52.481304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:12:52.481304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:12:52.481304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:12:52.481304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:12:52.481304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:12:52.481304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:12:52.481304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:12:52.481304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:12:52.484227 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 86/171] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:12:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:12:52.485232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b567bd2-771a-41c4-8e58-9dacf6bdc53d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:52.525907 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:52.610711 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b567bd2-771a-41c4-8e58-9dacf6bdc53d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:52.614552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 86/172] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:12:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:12:52.615362 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c268c-21b2-4ce0-8b3b-431234c7e24e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:52.653986 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d15a1-5239-4415-955f-c16ea28f6ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:52.655012 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d15a1-5239-4415-955f-c16ea28f6ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:52.656021 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d15a1-5239-4415-955f-c16ea28f6ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:52.656574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d15a1-5239-4415-955f-c16ea28f6ba9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:52.657806 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d15a1-5239-4415-955f-c16ea28f6ba9 None admin] RBAC: Adding query filter params (role.id=cfeec012f4ec48138c0e637e9a7fde61, scope.project.id=e54d8744be3c4672b24fa2e19f024af0, user.id=3181dca028754df5a6df1154d3635ddd) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:52.658346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d15a1-5239-4415-955f-c16ea28f6ba9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:52.660042 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d15a1-5239-4415-955f-c16ea28f6ba9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:52.668617 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 87/173] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:12:52 2022] GET /identity/v3/role_assignments?user.id=3181dca028754df5a6df1154d3635ddd&scope.project.id=e54d8744be3c4672b24fa2e19f024af0&role.id=cfeec012f4ec48138c0e637e9a7fde61 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:52.669823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d15a1-5239-4415-955f-c16ea28f6ba9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:52.799917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d15a1-5239-4415-955f-c16ea28f6ba9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:52.803868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:52.841051 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f2fe8-79ec-4ee0-a7e3-846fcd25f134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:52.842959 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f2fe8-79ec-4ee0-a7e3-846fcd25f134 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:52.842959 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f2fe8-79ec-4ee0-a7e3-846fcd25f134 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:52.843400 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f2fe8-79ec-4ee0-a7e3-846fcd25f134 None admin] PATH_INFO: `/v3/projects/e54d8744be3c4672b24fa2e19f024af0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:52.844860 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f2fe8-79ec-4ee0-a7e3-846fcd25f134 None admin] RBAC: Authorizing `identity:get_project(project_id=e54d8744be3c4672b24fa2e19f024af0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:52.845767 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c268c-21b2-4ce0-8b3b-431234c7e24e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:52.846812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4f2fe8-79ec-4ee0-a7e3-846fcd25f134 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:52.849139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 88/174] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:12:52 2022] GET /identity/v3/projects/e54d8744be3c4672b24fa2e19f024af0 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:52.855336 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f2fe8-79ec-4ee0-a7e3-846fcd25f134 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:52.901548 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa5410-5c44-48c2-9abe-cdefb78d180b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:52.902378 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa5410-5c44-48c2-9abe-cdefb78d180b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:52.902784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa5410-5c44-48c2-9abe-cdefb78d180b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:52.903068 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa5410-5c44-48c2-9abe-cdefb78d180b None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:52.903937 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa5410-5c44-48c2-9abe-cdefb78d180b None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:52.904312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa5410-5c44-48c2-9abe-cdefb78d180b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:52.905840 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa5410-5c44-48c2-9abe-cdefb78d180b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:52.907650 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:52.928145 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 87/175] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:12:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:52.934148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa5410-5c44-48c2-9abe-cdefb78d180b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:52.999879 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4f2fe8-79ec-4ee0-a7e3-846fcd25f134 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:53.044107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4dbca-5e5d-4373-ac04-f9e8d04853a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:53.045586 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4dbca-5e5d-4373-ac04-f9e8d04853a2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:53.046075 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4dbca-5e5d-4373-ac04-f9e8d04853a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:53.046552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4dbca-5e5d-4373-ac04-f9e8d04853a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:53.047621 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4dbca-5e5d-4373-ac04-f9e8d04853a2 None admin] RBAC: Adding query filter params (role.id=8acc34a163874a60b03728b814eaf075, scope.project.id=e54d8744be3c4672b24fa2e19f024af0, user.id=9242c0a59f6b49568f911aaa0ef688f2) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:53.048159 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4dbca-5e5d-4373-ac04-f9e8d04853a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:53.050000 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4dbca-5e5d-4373-ac04-f9e8d04853a2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:53.058790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 89/176] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:12:52 2022] GET /identity/v3/role_assignments?user.id=9242c0a59f6b49568f911aaa0ef688f2&scope.project.id=e54d8744be3c4672b24fa2e19f024af0&role.id=8acc34a163874a60b03728b814eaf075 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:53.124436 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa5410-5c44-48c2-9abe-cdefb78d180b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:53.178228 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c723d5ed-8963-487b-bee9-b23c0542ad5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:53.179117 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723d5ed-8963-487b-bee9-b23c0542ad5c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:53.179399 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723d5ed-8963-487b-bee9-b23c0542ad5c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:53.179693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c723d5ed-8963-487b-bee9-b23c0542ad5c None admin] PATH_INFO: `/v3/users/3181dca028754df5a6df1154d3635ddd` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:53.184343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723d5ed-8963-487b-bee9-b23c0542ad5c None admin] RBAC: Authorizing `identity:get_user(user_id=3181dca028754df5a6df1154d3635ddd)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:53.185846 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c723d5ed-8963-487b-bee9-b23c0542ad5c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:53.187593 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 88/177] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:12:52 2022] GET /identity/v3/users/3181dca028754df5a6df1154d3635ddd => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:12:53.193586 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4dbca-5e5d-4373-ac04-f9e8d04853a2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:53.245893 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:53.334989 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4dbca-5e5d-4373-ac04-f9e8d04853a2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:53.373937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0cb333-188b-47b8-abc2-ed8d65652722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:53.375029 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cb333-188b-47b8-abc2-ed8d65652722 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:53.375544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cb333-188b-47b8-abc2-ed8d65652722 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:53.376023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0cb333-188b-47b8-abc2-ed8d65652722 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:53.449131 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc681db7-8fc8-43ff-b387-fb45aac3c260 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:53.449436 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc681db7-8fc8-43ff-b387-fb45aac3c260 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:53.449694 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc681db7-8fc8-43ff-b387-fb45aac3c260 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:53.451080 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 89/178] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:53.452972 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0cb333-188b-47b8-abc2-ed8d65652722 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:53.454726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0cb333-188b-47b8-abc2-ed8d65652722 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:53.456923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2853a37-e3c3-4fc9-abb1-e242a82377c0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:53.457122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2853a37-e3c3-4fc9-abb1-e242a82377c0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:53.457404 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2853a37-e3c3-4fc9-abb1-e242a82377c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:53.495652 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d2853a37-e3c3-4fc9-abb1-e242a82377c0 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:53.496888 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d2853a37-e3c3-4fc9-abb1-e242a82377c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:53.499397 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:53.579377 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 90/179] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:12:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:12:53.584759 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0cb333-188b-47b8-abc2-ed8d65652722 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:53.638523 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:53.650260 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 90/180] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:12:53.652433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa6470-6ded-4d8c-b198-0bda330ceb94 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:53.652983 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa6470-6ded-4d8c-b198-0bda330ceb94 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:53.653269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa6470-6ded-4d8c-b198-0bda330ceb94 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:53.654569 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 91/181] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:53.658779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23215c-333c-4245-a76b-2fbdb84bc203 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:53.658779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23215c-333c-4245-a76b-2fbdb84bc203 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:53.658779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23215c-333c-4245-a76b-2fbdb84bc203 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:53.658779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 92/182] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:53.660096 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d8ea7-c26d-44a9-986b-fd895fcc4879 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:53.660385 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d8ea7-c26d-44a9-986b-fd895fcc4879 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:53.660709 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d8ea7-c26d-44a9-986b-fd895fcc4879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:53.683411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2c4d8ea7-c26d-44a9-986b-fd895fcc4879 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:53.684781 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2c4d8ea7-c26d-44a9-986b-fd895fcc4879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:53.724039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0cb333-188b-47b8-abc2-ed8d65652722 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:53.755829 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:53.763253 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045463d-239d-4e4f-9100-8c336947263e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:53.764144 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045463d-239d-4e4f-9100-8c336947263e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:53.769532 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 93/183] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:12:53.769631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045463d-239d-4e4f-9100-8c336947263e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:53.769631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045463d-239d-4e4f-9100-8c336947263e None admin] PATH_INFO: `/v3/projects/54d37e8eb5424296b3a4c9602e06061d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:53.769631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045463d-239d-4e4f-9100-8c336947263e None admin] RBAC: Authorizing `identity:get_project(project_id=54d37e8eb5424296b3a4c9602e06061d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:53.769631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045463d-239d-4e4f-9100-8c336947263e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:53.769631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 91/184] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:12:53 2022] GET /identity/v3/projects/54d37e8eb5424296b3a4c9602e06061d => generated 312 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:12:53.778541 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf512d-09da-4722-8882-2441042bf28f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:53.778541 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf512d-09da-4722-8882-2441042bf28f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:53.778541 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf512d-09da-4722-8882-2441042bf28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:53.778736 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045463d-239d-4e4f-9100-8c336947263e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:53.788533 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7edf512d-09da-4722-8882-2441042bf28f None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:53.790271 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7edf512d-09da-4722-8882-2441042bf28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:53.868237 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 94/185] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:12:53.868950 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf512d-09da-4722-8882-2441042bf28f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:53.884777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045463d-239d-4e4f-9100-8c336947263e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:53.923259 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449027f1-434b-4b11-9a53-bbb93820e543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:53.924364 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449027f1-434b-4b11-9a53-bbb93820e543 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:53.924643 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449027f1-434b-4b11-9a53-bbb93820e543 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:53.924913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449027f1-434b-4b11-9a53-bbb93820e543 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:53.938988 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449027f1-434b-4b11-9a53-bbb93820e543 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:12:53.942147 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:53.949856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-449027f1-434b-4b11-9a53-bbb93820e543 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:12:53.949856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:12:53.949856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:12:53.949856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:12:53.949856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:12:53.949856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:12:53.949856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:12:53.949856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:12:53.949856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:12:53.949856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:12:53.949856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:12:53.949856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:12:53.949856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:12:53.949856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:12:53.949856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:12:53.949856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:12:53.950889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:12:53.950889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:12:53.950889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:12:53.950889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:12:53.950889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:12:53.950889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:12:53.950889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:12:53.950889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:12:53.950889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:12:53.950889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:12:53.950889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:12:53.950889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:12:53.950889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:12:53.950889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:12:53.950889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:12:53.950889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:12:53.951729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:12:53.951729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:12:53.951729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:12:53.951729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:12:53.951729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:12:53.951729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:12:53.951729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:12:53.951729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:12:53.951729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:12:53.951729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:12:53.951729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:12:53.951729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:12:53.951729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 92/186] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:12:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:12:53.952386 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449027f1-434b-4b11-9a53-bbb93820e543 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:53.991650 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf512d-09da-4722-8882-2441042bf28f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:54.022041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:54.038177 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b84221-a938-4749-ace5-f3d69a337385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:54.038857 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b84221-a938-4749-ace5-f3d69a337385 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:54.039151 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b84221-a938-4749-ace5-f3d69a337385 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:54.039432 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b84221-a938-4749-ace5-f3d69a337385 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:54.040645 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b84221-a938-4749-ace5-f3d69a337385 None admin] RBAC: Adding query filter params (role.id=9ab3e302ead54950b7c30103f1e7fda7, scope.project.id=54d37e8eb5424296b3a4c9602e06061d, user.id=3181dca028754df5a6df1154d3635ddd) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:54.040956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b84221-a938-4749-ace5-f3d69a337385 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:54.042482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b84221-a938-4749-ace5-f3d69a337385 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:54.051085 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 95/187] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:12:53 2022] GET /identity/v3/role_assignments?user.id=3181dca028754df5a6df1154d3635ddd&scope.project.id=54d37e8eb5424296b3a4c9602e06061d&role.id=9ab3e302ead54950b7c30103f1e7fda7 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:54.052018 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b84221-a938-4749-ace5-f3d69a337385 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:54.079137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449027f1-434b-4b11-9a53-bbb93820e543 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:54.124691 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733af973-d886-4285-a5a4-5353f2a33f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:54.125551 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733af973-d886-4285-a5a4-5353f2a33f71 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:54.125851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733af973-d886-4285-a5a4-5353f2a33f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:54.126162 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733af973-d886-4285-a5a4-5353f2a33f71 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:54.127393 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733af973-d886-4285-a5a4-5353f2a33f71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:54.128934 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733af973-d886-4285-a5a4-5353f2a33f71 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:54.130871 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 93/188] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:12:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:54.131578 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733af973-d886-4285-a5a4-5353f2a33f71 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:54.193836 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b84221-a938-4749-ace5-f3d69a337385 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:54.229290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:54.237955 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891cb7df-e71b-4df1-adb1-9ca9d5ec2eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:54.239204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891cb7df-e71b-4df1-adb1-9ca9d5ec2eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:54.239561 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891cb7df-e71b-4df1-adb1-9ca9d5ec2eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:54.239807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891cb7df-e71b-4df1-adb1-9ca9d5ec2eb1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:54.259277 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891cb7df-e71b-4df1-adb1-9ca9d5ec2eb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:12:54.264581 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733af973-d886-4285-a5a4-5353f2a33f71 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:54.268783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-891cb7df-e71b-4df1-adb1-9ca9d5ec2eb1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:12:54.268783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:12:54.268783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:12:54.268783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:12:54.268783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:12:54.268783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:12:54.268783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:12:54.268783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:12:54.268783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:12:54.268783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:12:54.268783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:12:54.268783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:12:54.268783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:12:54.268783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:12:54.268783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:12:54.268783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:12:54.269849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:12:54.269849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:12:54.269849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:12:54.269849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:12:54.269849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:12:54.269849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:12:54.269849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:12:54.269849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:12:54.269849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:12:54.269849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:12:54.269849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:12:54.269849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:12:54.269849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:12:54.269849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:12:54.269849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:12:54.269849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:12:54.270819 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:12:54.270819 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:12:54.270819 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:12:54.270819 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:12:54.270819 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:12:54.270819 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:12:54.270819 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:12:54.270819 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:12:54.270819 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:12:54.270819 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:12:54.270819 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:12:54.270819 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:12:54.270819 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 96/189] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 30 16:12:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:12:54.271653 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891cb7df-e71b-4df1-adb1-9ca9d5ec2eb1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:54.301894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:54.315192 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c780f-27bb-4877-b07b-6bf15a900df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:54.316109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c780f-27bb-4877-b07b-6bf15a900df2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:54.316539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c780f-27bb-4877-b07b-6bf15a900df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:54.317000 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c780f-27bb-4877-b07b-6bf15a900df2 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:54.318995 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4c780f-27bb-4877-b07b-6bf15a900df2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:54.322574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4c780f-27bb-4877-b07b-6bf15a900df2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:54.322574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4c780f-27bb-4877-b07b-6bf15a900df2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:54.330946 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 94/190] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:12:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:54.332102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c780f-27bb-4877-b07b-6bf15a900df2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:54.383777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891cb7df-e71b-4df1-adb1-9ca9d5ec2eb1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:54.422026 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043c853-a12f-452b-8ae3-2dc3becb4034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:54.422701 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043c853-a12f-452b-8ae3-2dc3becb4034 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:54.422987 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043c853-a12f-452b-8ae3-2dc3becb4034 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:54.423272 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043c853-a12f-452b-8ae3-2dc3becb4034 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:54.424023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043c853-a12f-452b-8ae3-2dc3becb4034 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:54.425483 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043c853-a12f-452b-8ae3-2dc3becb4034 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:54.441528 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c780f-27bb-4877-b07b-6bf15a900df2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:54.475080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0bde7f-c556-4105-b5ed-4a0a6380ff1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:54.475685 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bde7f-c556-4105-b5ed-4a0a6380ff1c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:54.475970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bde7f-c556-4105-b5ed-4a0a6380ff1c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:54.476271 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0bde7f-c556-4105-b5ed-4a0a6380ff1c None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:54.476995 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0bde7f-c556-4105-b5ed-4a0a6380ff1c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:54.477819 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0bde7f-c556-4105-b5ed-4a0a6380ff1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:54.479199 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0bde7f-c556-4105-b5ed-4a0a6380ff1c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:54.487430 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 95/191] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:12:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:54.488234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0bde7f-c556-4105-b5ed-4a0a6380ff1c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:54.536083 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:54.576114 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 97/192] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 30 16:12:54 2022] POST /identity/v3/users => generated 297 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:12:54.577371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043c853-a12f-452b-8ae3-2dc3becb4034 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:54.608503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0bde7f-c556-4105-b5ed-4a0a6380ff1c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:54.627763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:54.647795 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9ec80-36cd-4b75-89c8-1192dcd26f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:54.648720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9ec80-36cd-4b75-89c8-1192dcd26f31 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:54.649499 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9ec80-36cd-4b75-89c8-1192dcd26f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:54.649782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9ec80-36cd-4b75-89c8-1192dcd26f31 None admin] PATH_INFO: `/v3/users/62bf7c2b073e4fe783f3ebeb3fa59fef` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:54.668144 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9ec80-36cd-4b75-89c8-1192dcd26f31 None admin] RBAC: Authorizing `identity:get_user(user_id=62bf7c2b073e4fe783f3ebeb3fa59fef)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:54.669639 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9ec80-36cd-4b75-89c8-1192dcd26f31 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:54.676009 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 96/193] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:12:54 2022] GET /identity/v3/users/62bf7c2b073e4fe783f3ebeb3fa59fef => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:54.676009 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b84db-0889-4f8f-971c-62047584d53e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:54.676009 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b84db-0889-4f8f-971c-62047584d53e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:54.676009 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b84db-0889-4f8f-971c-62047584d53e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:54.676009 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 97/194] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:54.679649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b84db-0889-4f8f-971c-62047584d53e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:54.710356 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043c853-a12f-452b-8ae3-2dc3becb4034 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:54.752943 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497fb3f6-9eb1-424b-9632-ce902503a4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:54.753774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fb3f6-9eb1-424b-9632-ce902503a4ff None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:54.754060 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fb3f6-9eb1-424b-9632-ce902503a4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:54.754336 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fb3f6-9eb1-424b-9632-ce902503a4ff None admin] PATH_INFO: `/v3/users/62bf7c2b073e4fe783f3ebeb3fa59fef` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:54.756178 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497fb3f6-9eb1-424b-9632-ce902503a4ff None admin] RBAC: Authorizing `identity:get_user(user_id=62bf7c2b073e4fe783f3ebeb3fa59fef)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:54.757663 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497fb3f6-9eb1-424b-9632-ce902503a4ff None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:54.759416 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 98/195] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:12:54 2022] GET /identity/v3/users/62bf7c2b073e4fe783f3ebeb3fa59fef => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:54.760630 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c8f62-f466-4e87-ad99-35fabfc585df None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:54.760919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c8f62-f466-4e87-ad99-35fabfc585df None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:54.761194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58c8f62-f466-4e87-ad99-35fabfc585df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:54.779644 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f58c8f62-f466-4e87-ad99-35fabfc585df None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:54.780814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f58c8f62-f466-4e87-ad99-35fabfc585df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:54.807606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8b84db-0889-4f8f-971c-62047584d53e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:54.815432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:54.850576 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5546980b-f6f2-4fa1-869d-5c8039ada1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:54.851318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5546980b-f6f2-4fa1-869d-5c8039ada1de None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:54.851706 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5546980b-f6f2-4fa1-869d-5c8039ada1de None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:54.851986 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5546980b-f6f2-4fa1-869d-5c8039ada1de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:54.871868 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 99/196] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:12:54.872751 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c8f62-f466-4e87-ad99-35fabfc585df None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:54.910223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:54.942025 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5546980b-f6f2-4fa1-869d-5c8039ada1de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:54.944071 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5546980b-f6f2-4fa1-869d-5c8039ada1de None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:54.990498 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58c8f62-f466-4e87-ad99-35fabfc585df None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:55.034919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486cf27-903e-4906-b6c3-9e2126aaafd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:55.035500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486cf27-903e-4906-b6c3-9e2126aaafd4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:55.035815 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486cf27-903e-4906-b6c3-9e2126aaafd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:55.036098 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6486cf27-903e-4906-b6c3-9e2126aaafd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:55.081931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 98/197] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:12:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:12:55.082794 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5546980b-f6f2-4fa1-869d-5c8039ada1de None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:55.097437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:55.127920 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486cf27-903e-4906-b6c3-9e2126aaafd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:55.129437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6486cf27-903e-4906-b6c3-9e2126aaafd4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:55.173115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:55.202556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5546980b-f6f2-4fa1-869d-5c8039ada1de None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:55.243713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e685ef-edd1-4f15-a0c9-dd9c051f9cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:55.244679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e685ef-edd1-4f15-a0c9-dd9c051f9cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:55.245131 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e685ef-edd1-4f15-a0c9-dd9c051f9cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:55.245486 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e685ef-edd1-4f15-a0c9-dd9c051f9cc0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:55.257052 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e685ef-edd1-4f15-a0c9-dd9c051f9cc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:12:55.265848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-75e685ef-edd1-4f15-a0c9-dd9c051f9cc0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 16:12:55.265848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:12:55.265848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:12:55.265848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:12:55.265848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:12:55.265848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:12:55.265848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:12:55.265848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:12:55.265848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:12:55.265848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:12:55.265848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:12:55.265848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:12:55.265848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:12:55.265848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:12:55.265848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:12:55.265848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:12:55.266933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:12:55.266933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:12:55.266933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:12:55.266933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:12:55.266933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:12:55.266933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:12:55.266933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:12:55.266933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:12:55.266933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:12:55.266933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:12:55.266933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:12:55.266933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:12:55.266933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:12:55.266933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:12:55.266933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:12:55.266933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:12:55.267926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:12:55.267926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:12:55.267926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:12:55.267926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:12:55.267926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:12:55.267926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:12:55.267926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:12:55.267926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:12:55.267926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:12:55.267926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:12:55.267926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 16:12:55.267926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:12:55.269239 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 99/198] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:12:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:12:55.269611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 100/199] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:12:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:12:55.274483 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e685ef-edd1-4f15-a0c9-dd9c051f9cc0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:55.277007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486cf27-903e-4906-b6c3-9e2126aaafd4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:55.350273 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:55.383101 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6486cf27-903e-4906-b6c3-9e2126aaafd4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:55.390275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e685ef-edd1-4f15-a0c9-dd9c051f9cc0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:55.415190 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:55.420455 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55d29b-5fb6-4e60-97bd-066bdd29624f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:55.421326 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55d29b-5fb6-4e60-97bd-066bdd29624f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:55.421681 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55d29b-5fb6-4e60-97bd-066bdd29624f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:55.422034 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55d29b-5fb6-4e60-97bd-066bdd29624f None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:55.422865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a55d29b-5fb6-4e60-97bd-066bdd29624f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:55.423248 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a55d29b-5fb6-4e60-97bd-066bdd29624f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:55.425000 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a55d29b-5fb6-4e60-97bd-066bdd29624f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:55.430704 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a8e0a-1353-4d8e-87fb-082290eb67d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:55.431895 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a8e0a-1353-4d8e-87fb-082290eb67d6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:55.432362 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a8e0a-1353-4d8e-87fb-082290eb67d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:55.432833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a8e0a-1353-4d8e-87fb-082290eb67d6 None admin] PATH_INFO: `/v3/projects/195eb382a49146b6b27a5cdc500c1a05` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:55.434275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a8e0a-1353-4d8e-87fb-082290eb67d6 None admin] RBAC: Authorizing `identity:get_project(project_id=195eb382a49146b6b27a5cdc500c1a05)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:55.435989 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a8e0a-1353-4d8e-87fb-082290eb67d6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:55.438025 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 100/200] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:12:55 2022] GET /identity/v3/projects/195eb382a49146b6b27a5cdc500c1a05 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:55.439256 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a8e0a-1353-4d8e-87fb-082290eb67d6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:55.440518 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 101/201] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:12:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:55.441427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55d29b-5fb6-4e60-97bd-066bdd29624f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:55.562038 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a8e0a-1353-4d8e-87fb-082290eb67d6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:55.578541 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55d29b-5fb6-4e60-97bd-066bdd29624f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:55.600247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59ed46-75e5-4c0b-a767-4e3e48be3f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:55.601426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59ed46-75e5-4c0b-a767-4e3e48be3f86 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:55.601664 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59ed46-75e5-4c0b-a767-4e3e48be3f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:55.602004 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59ed46-75e5-4c0b-a767-4e3e48be3f86 None admin] PATH_INFO: `/v3/projects/195eb382a49146b6b27a5cdc500c1a05` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:55.603499 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59ed46-75e5-4c0b-a767-4e3e48be3f86 None admin] RBAC: Authorizing `identity:get_project(project_id=195eb382a49146b6b27a5cdc500c1a05)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:55.605097 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59ed46-75e5-4c0b-a767-4e3e48be3f86 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:55.607444 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 101/202] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:12:55 2022] GET /identity/v3/projects/195eb382a49146b6b27a5cdc500c1a05 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:55.608403 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59ed46-75e5-4c0b-a767-4e3e48be3f86 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:55.611282 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:55.623165 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4decb306-71d2-49b1-aa19-9c5399a8106c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:55.624581 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decb306-71d2-49b1-aa19-9c5399a8106c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:55.625011 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decb306-71d2-49b1-aa19-9c5399a8106c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:55.626777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decb306-71d2-49b1-aa19-9c5399a8106c None admin] PATH_INFO: `/v3/users/2f71ac02e31149d6a110341ff612d852` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:55.627497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4decb306-71d2-49b1-aa19-9c5399a8106c None admin] RBAC: Authorizing `identity:get_user(user_id=2f71ac02e31149d6a110341ff612d852)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:55.629381 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4decb306-71d2-49b1-aa19-9c5399a8106c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:55.631454 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 102/203] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:12:55 2022] GET /identity/v3/users/2f71ac02e31149d6a110341ff612d852 => generated 289 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:55.632379 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4decb306-71d2-49b1-aa19-9c5399a8106c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:55.720110 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:55.725871 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59ed46-75e5-4c0b-a767-4e3e48be3f86 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:55.769958 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c008b51-7bad-480b-af6d-bb6bbbe0b519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:55.770893 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c008b51-7bad-480b-af6d-bb6bbbe0b519 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:55.771529 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c008b51-7bad-480b-af6d-bb6bbbe0b519 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:55.772335 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c008b51-7bad-480b-af6d-bb6bbbe0b519 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:55.773617 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c008b51-7bad-480b-af6d-bb6bbbe0b519 None admin] RBAC: Adding query filter params (role.id=8acc34a163874a60b03728b814eaf075, scope.project.id=195eb382a49146b6b27a5cdc500c1a05, user.id=62bf7c2b073e4fe783f3ebeb3fa59fef) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:55.774102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c008b51-7bad-480b-af6d-bb6bbbe0b519 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:55.775317 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4decb306-71d2-49b1-aa19-9c5399a8106c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:55.776229 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c008b51-7bad-480b-af6d-bb6bbbe0b519 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:55.784430 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 102/204] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:12:55 2022] GET /identity/v3/role_assignments?user.id=62bf7c2b073e4fe783f3ebeb3fa59fef&scope.project.id=195eb382a49146b6b27a5cdc500c1a05&role.id=8acc34a163874a60b03728b814eaf075 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:55.785440 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c008b51-7bad-480b-af6d-bb6bbbe0b519 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:55.849885 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a7c46-db32-43a2-8112-fe5b0e400344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:55.850708 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a7c46-db32-43a2-8112-fe5b0e400344 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:55.851363 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a7c46-db32-43a2-8112-fe5b0e400344 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:55.851556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a7c46-db32-43a2-8112-fe5b0e400344 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:55.853063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a7c46-db32-43a2-8112-fe5b0e400344 None admin] RBAC: Adding query filter params (role.id=cfeec012f4ec48138c0e637e9a7fde61, scope.project.id=195eb382a49146b6b27a5cdc500c1a05, user.id=62bf7c2b073e4fe783f3ebeb3fa59fef) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:55.853510 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a7c46-db32-43a2-8112-fe5b0e400344 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:55.855171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a7c46-db32-43a2-8112-fe5b0e400344 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:55.870431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 103/205] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:12:55 2022] GET /identity/v3/role_assignments?user.id=62bf7c2b073e4fe783f3ebeb3fa59fef&scope.project.id=195eb382a49146b6b27a5cdc500c1a05&role.id=cfeec012f4ec48138c0e637e9a7fde61 => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:55.871572 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:55.920281 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c008b51-7bad-480b-af6d-bb6bbbe0b519 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:55.960031 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d58a0-d4eb-4a6f-bdc1-0f73ae24c50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:55.960946 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d58a0-d4eb-4a6f-bdc1-0f73ae24c50d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:55.961411 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d58a0-d4eb-4a6f-bdc1-0f73ae24c50d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:55.961861 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d58a0-d4eb-4a6f-bdc1-0f73ae24c50d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:56.045376 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d58a0-d4eb-4a6f-bdc1-0f73ae24c50d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:56.046890 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d58a0-d4eb-4a6f-bdc1-0f73ae24c50d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:56.118668 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:56.155285 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 103/206] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:12:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:12:56.161276 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a7c46-db32-43a2-8112-fe5b0e400344 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:56.307818 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a7c46-db32-43a2-8112-fe5b0e400344 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:56.346858 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:56.356602 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e3370e-1c88-4b49-adc4-165b7a70c501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:56.357497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3370e-1c88-4b49-adc4-165b7a70c501 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:56.362929 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3370e-1c88-4b49-adc4-165b7a70c501 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:56.362929 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e3370e-1c88-4b49-adc4-165b7a70c501 None admin] PATH_INFO: `/v3/projects/e54d8744be3c4672b24fa2e19f024af0` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:56.364298 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e3370e-1c88-4b49-adc4-165b7a70c501 None admin] RBAC: Authorizing `identity:get_project(project_id=e54d8744be3c4672b24fa2e19f024af0)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:56.365813 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e3370e-1c88-4b49-adc4-165b7a70c501 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:56.368144 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 104/207] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:12:56 2022] GET /identity/v3/projects/e54d8744be3c4672b24fa2e19f024af0 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:56.374416 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d58a0-d4eb-4a6f-bdc1-0f73ae24c50d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:56.504922 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d58a0-d4eb-4a6f-bdc1-0f73ae24c50d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:56.550154 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e21c69-6f8e-453e-9b99-57b1a81345e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:56.550821 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e21c69-6f8e-453e-9b99-57b1a81345e4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:56.551103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e21c69-6f8e-453e-9b99-57b1a81345e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:56.551378 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e21c69-6f8e-453e-9b99-57b1a81345e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:56.552233 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e21c69-6f8e-453e-9b99-57b1a81345e4 None admin] RBAC: Adding query filter params (role.id=6e5a95d194ca42cab7a30baff4554fd3, scope.project.id=e54d8744be3c4672b24fa2e19f024af0, user.id=2f71ac02e31149d6a110341ff612d852) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:56.552546 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e21c69-6f8e-453e-9b99-57b1a81345e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:56.554002 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e21c69-6f8e-453e-9b99-57b1a81345e4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:56.561040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 104/208] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:12:56 2022] GET /identity/v3/role_assignments?user.id=2f71ac02e31149d6a110341ff612d852&scope.project.id=e54d8744be3c4672b24fa2e19f024af0&role.id=6e5a95d194ca42cab7a30baff4554fd3 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:56.911112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38e6e8-7f87-4ea2-a4d7-86ffa46442fd None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:56.911504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38e6e8-7f87-4ea2-a4d7-86ffa46442fd None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:56.911781 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38e6e8-7f87-4ea2-a4d7-86ffa46442fd None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:56.913374 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 105/209] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:56.919074 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7e162-3edf-4fd6-a010-eae982c31096 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:56.919371 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7e162-3edf-4fd6-a010-eae982c31096 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:56.919686 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7e162-3edf-4fd6-a010-eae982c31096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:56.938558 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d5c7e162-3edf-4fd6-a010-eae982c31096 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:56.939843 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d5c7e162-3edf-4fd6-a010-eae982c31096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:56.966690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:57.013909 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 105/210] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:12:57.038034 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38e6e8-7f87-4ea2-a4d7-86ffa46442fd None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:57.154928 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38e6e8-7f87-4ea2-a4d7-86ffa46442fd None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:57.194298 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a96cd-71e4-4855-9ab2-62ba4595294f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:57.195044 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a96cd-71e4-4855-9ab2-62ba4595294f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:57.195363 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a96cd-71e4-4855-9ab2-62ba4595294f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:57.196283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a96cd-71e4-4855-9ab2-62ba4595294f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:57.208405 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a96cd-71e4-4855-9ab2-62ba4595294f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:12:57.217189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-355a96cd-71e4-4855-9ab2-62ba4595294f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:12:57.217189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:12:57.217189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:12:57.217189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:12:57.217189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:12:57.217189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:12:57.217189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:12:57.217189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:12:57.217189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:12:57.217189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:12:57.217189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:12:57.217189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:12:57.217189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:12:57.217189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:12:57.217189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:12:57.217189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:12:57.218241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:12:57.218241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:12:57.218241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:12:57.218241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:12:57.218241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:12:57.218241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:12:57.218241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:12:57.218241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:12:57.218241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:12:57.218241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:12:57.218241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:12:57.218241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:12:57.218241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:12:57.218241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:12:57.218241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:12:57.218241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:12:57.219013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:12:57.219013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:12:57.219013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:12:57.219013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:12:57.219013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:12:57.219013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:12:57.219013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:12:57.219013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:12:57.219013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:12:57.219013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:12:57.219013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:12:57.219013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:12:57.219013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 106/211] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:12:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:12:57.228333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7e162-3edf-4fd6-a010-eae982c31096 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:57.305305 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fb8ea-a9cb-4d08-b2ee-96a7560ba054 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:57.305621 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fb8ea-a9cb-4d08-b2ee-96a7560ba054 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:57.305922 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fb8ea-a9cb-4d08-b2ee-96a7560ba054 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:57.307456 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 107/212] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:12:57.313227 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27337dd-49e8-4220-8955-16c4815b0218 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:57.313807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27337dd-49e8-4220-8955-16c4815b0218 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:57.314681 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27337dd-49e8-4220-8955-16c4815b0218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:57.338104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a27337dd-49e8-4220-8955-16c4815b0218 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:57.338554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7e162-3edf-4fd6-a010-eae982c31096 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:57.340123 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a27337dd-49e8-4220-8955-16c4815b0218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:57.373970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1da558-f8b7-44ca-b4e6-729eeca0477d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:57.374665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1da558-f8b7-44ca-b4e6-729eeca0477d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:57.375026 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1da558-f8b7-44ca-b4e6-729eeca0477d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:57.375231 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1da558-f8b7-44ca-b4e6-729eeca0477d None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:57.376079 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1da558-f8b7-44ca-b4e6-729eeca0477d None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:57.376390 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1da558-f8b7-44ca-b4e6-729eeca0477d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:57.377896 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1da558-f8b7-44ca-b4e6-729eeca0477d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:57.392852 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 106/213] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:12:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:57.394047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1da558-f8b7-44ca-b4e6-729eeca0477d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:57.419934 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:57.427083 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:57.429978 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 108/214] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:12:57.455943 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27337dd-49e8-4220-8955-16c4815b0218 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:57.573084 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1da558-f8b7-44ca-b4e6-729eeca0477d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:57.618805 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27337dd-49e8-4220-8955-16c4815b0218 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:57.619089 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc469c-46da-46b4-93a9-f55575a0430a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:57.620145 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc469c-46da-46b4-93a9-f55575a0430a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:57.620822 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc469c-46da-46b4-93a9-f55575a0430a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:57.621253 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc469c-46da-46b4-93a9-f55575a0430a None admin] PATH_INFO: `/v3/users/0f9c97c3ab734e1f9f553ccd41e81008` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:57.640041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc469c-46da-46b4-93a9-f55575a0430a None admin] RBAC: Authorizing `identity:get_user(user_id=0f9c97c3ab734e1f9f553ccd41e81008)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:57.642435 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadc469c-46da-46b4-93a9-f55575a0430a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:57.649153 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 107/215] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:12:57 2022] GET /identity/v3/users/0f9c97c3ab734e1f9f553ccd41e81008 => generated 297 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:57.651375 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc469c-46da-46b4-93a9-f55575a0430a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:57.668912 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9212e8-7e71-4afa-ae9f-77ce7a659114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:57.670049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9212e8-7e71-4afa-ae9f-77ce7a659114 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:57.670376 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9212e8-7e71-4afa-ae9f-77ce7a659114 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:57.670688 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9212e8-7e71-4afa-ae9f-77ce7a659114 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:57.672210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9212e8-7e71-4afa-ae9f-77ce7a659114 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:57.674232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9212e8-7e71-4afa-ae9f-77ce7a659114 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:57.681656 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 109/216] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:12:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:12:57.684467 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9212e8-7e71-4afa-ae9f-77ce7a659114 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:57.741650 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:57.747027 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:57.767597 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc469c-46da-46b4-93a9-f55575a0430a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:57.796217 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9212e8-7e71-4afa-ae9f-77ce7a659114 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:57.804183 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ef1c8-5e54-4fb1-b122-f88f70672fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:57.804845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ef1c8-5e54-4fb1-b122-f88f70672fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:57.805650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ef1c8-5e54-4fb1-b122-f88f70672fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:57.805926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ef1c8-5e54-4fb1-b122-f88f70672fd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:57.836129 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ebb92-2358-462a-ad87-fdc6e5de0bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:57.837307 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ebb92-2358-462a-ad87-fdc6e5de0bba None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:57.837799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ebb92-2358-462a-ad87-fdc6e5de0bba None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:57.838313 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ebb92-2358-462a-ad87-fdc6e5de0bba None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:57.839234 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ebb92-2358-462a-ad87-fdc6e5de0bba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:57.840979 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ebb92-2358-462a-ad87-fdc6e5de0bba None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:57.879301 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ef1c8-5e54-4fb1-b122-f88f70672fd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:57.880808 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092ef1c8-5e54-4fb1-b122-f88f70672fd8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:57.965075 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:57.986740 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 110/217] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 30 16:12:57 2022] POST /identity/v3/users => generated 297 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:12:57.995716 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 108/218] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:12:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:12:58.002296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ef1c8-5e54-4fb1-b122-f88f70672fd8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:58.125291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ef1c8-5e54-4fb1-b122-f88f70672fd8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:58.144639 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6eac2-265b-47e5-91f9-310f8bbe7268 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:58.144959 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6eac2-265b-47e5-91f9-310f8bbe7268 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:58.145215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6eac2-265b-47e5-91f9-310f8bbe7268 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:58.146603 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 111/219] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:58.153137 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2c8e1-b231-421f-b7ce-25b48095b570 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:58.153426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2c8e1-b231-421f-b7ce-25b48095b570 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:58.153708 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2c8e1-b231-421f-b7ce-25b48095b570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:58.174286 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c45e62-d1f6-4f9d-a43b-6da82b5d9884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:58.175156 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c45e62-d1f6-4f9d-a43b-6da82b5d9884 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:58.175441 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c45e62-d1f6-4f9d-a43b-6da82b5d9884 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:58.175771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c45e62-d1f6-4f9d-a43b-6da82b5d9884 None admin] PATH_INFO: `/v3/projects/195eb382a49146b6b27a5cdc500c1a05` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:58.177032 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ace2c8e1-b231-421f-b7ce-25b48095b570 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:58.177978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c45e62-d1f6-4f9d-a43b-6da82b5d9884 None admin] RBAC: Authorizing `identity:get_project(project_id=195eb382a49146b6b27a5cdc500c1a05)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:58.178165 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ace2c8e1-b231-421f-b7ce-25b48095b570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:58.179446 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c45e62-d1f6-4f9d-a43b-6da82b5d9884 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:58.181626 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 109/220] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:12:58 2022] GET /identity/v3/projects/195eb382a49146b6b27a5cdc500c1a05 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:58.187346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c45e62-d1f6-4f9d-a43b-6da82b5d9884 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:58.250802 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:58.263416 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 112/221] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:12:58.287000 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2c8e1-b231-421f-b7ce-25b48095b570 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:58.323365 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c45e62-d1f6-4f9d-a43b-6da82b5d9884 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:58.335973 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:58.369249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e16a4-4cef-440b-aae1-276c385bd1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:58.369452 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e16a4-4cef-440b-aae1-276c385bd1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:58.369704 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e16a4-4cef-440b-aae1-276c385bd1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:58.369982 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e16a4-4cef-440b-aae1-276c385bd1c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:58.370820 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e16a4-4cef-440b-aae1-276c385bd1c7 None admin] RBAC: Adding query filter params (role.id=9ab3e302ead54950b7c30103f1e7fda7, scope.project.id=195eb382a49146b6b27a5cdc500c1a05, user.id=0f9c97c3ab734e1f9f553ccd41e81008) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:58.371139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e16a4-4cef-440b-aae1-276c385bd1c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:58.373730 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3e16a4-4cef-440b-aae1-276c385bd1c7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:58.387926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 110/222] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:12:58 2022] GET /identity/v3/role_assignments?user.id=0f9c97c3ab734e1f9f553ccd41e81008&scope.project.id=195eb382a49146b6b27a5cdc500c1a05&role.id=9ab3e302ead54950b7c30103f1e7fda7 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:12:58.389318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb05fe-5db6-4d1c-9e83-8e1c11aa8977 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:58.389558 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb05fe-5db6-4d1c-9e83-8e1c11aa8977 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:58.389835 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb05fe-5db6-4d1c-9e83-8e1c11aa8977 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:58.391105 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 111/223] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:12:58.397699 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf9028-42c5-4899-a6d3-5f76b4b300c5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:58.398101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf9028-42c5-4899-a6d3-5f76b4b300c5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:58.398447 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bf9028-42c5-4899-a6d3-5f76b4b300c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:58.418142 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b7bf9028-42c5-4899-a6d3-5f76b4b300c5 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:12:58.419813 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b7bf9028-42c5-4899-a6d3-5f76b4b300c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:12:58.431545 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2c8e1-b231-421f-b7ce-25b48095b570 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:58.470528 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc99545-a407-4406-9b0d-ffeed212bb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:58.471362 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc99545-a407-4406-9b0d-ffeed212bb5f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:58.471670 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc99545-a407-4406-9b0d-ffeed212bb5f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:58.471958 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc99545-a407-4406-9b0d-ffeed212bb5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:58.486044 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc99545-a407-4406-9b0d-ffeed212bb5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:12:58.495063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ffc99545-a407-4406-9b0d-ffeed212bb5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:12:58.495063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:12:58.495063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:12:58.495063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:12:58.495063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:12:58.495063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:12:58.495063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:12:58.495063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:12:58.495063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:12:58.495063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:12:58.495063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:12:58.495063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:12:58.495063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:12:58.495063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:12:58.495063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:12:58.495063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:12:58.496061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:12:58.496061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:12:58.496061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:12:58.496061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:12:58.496061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:12:58.496061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:12:58.496061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:12:58.496061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:12:58.496061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:12:58.496061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:12:58.496061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:12:58.496061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:12:58.496061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:12:58.496061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:12:58.496061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:12:58.496061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:12:58.496831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:12:58.496831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:12:58.496831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:12:58.496831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:12:58.496831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:12:58.496831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:12:58.496831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:12:58.496831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:12:58.496831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:12:58.496831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:12:58.496831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:12:58.496831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:12:58.496831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 113/224] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:12:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:12:58.502654 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc99545-a407-4406-9b0d-ffeed212bb5f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:58.590211 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 112/225] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:12:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:12:58.613844 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf9028-42c5-4899-a6d3-5f76b4b300c5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:58.676639 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:58.731840 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:58.736116 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc99545-a407-4406-9b0d-ffeed212bb5f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:58.751118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bf9028-42c5-4899-a6d3-5f76b4b300c5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:58.791307 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8db1e9-01f8-41e7-b912-bc5f30222c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:58.792400 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8db1e9-01f8-41e7-b912-bc5f30222c7e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:58.793008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8db1e9-01f8-41e7-b912-bc5f30222c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:58.793426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8db1e9-01f8-41e7-b912-bc5f30222c7e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:58.807307 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8db1e9-01f8-41e7-b912-bc5f30222c7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:12:58.816928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ea8db1e9-01f8-41e7-b912-bc5f30222c7e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:12:58.816928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:12:58.816928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:12:58.816928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:12:58.816928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:12:58.816928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:12:58.816928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:12:58.816928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:12:58.816928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:12:58.816928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:12:58.816928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:12:58.816928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:12:58.816928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:12:58.816928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:12:58.816928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:12:58.816928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:12:58.818001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:12:58.818001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:12:58.818001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:12:58.818001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:12:58.818001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:12:58.818001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:12:58.818001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:12:58.818001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:12:58.818001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:12:58.818001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:12:58.818001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:12:58.818001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:12:58.818001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:12:58.818001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:12:58.818001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:12:58.818001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:12:58.818965 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:12:58.818965 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:12:58.818965 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:12:58.818965 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:12:58.818965 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:12:58.818965 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:12:58.818965 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:12:58.818965 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:12:58.818965 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:12:58.818965 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:12:58.818965 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:12:58.818965 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:12:58.821819 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 113/226] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 30 16:12:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:12:58.827890 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8db1e9-01f8-41e7-b912-bc5f30222c7e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:58.832815 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9dd32-94bc-4807-9ed7-a5c7d45321c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:58.833728 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9dd32-94bc-4807-9ed7-a5c7d45321c3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:58.835877 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9dd32-94bc-4807-9ed7-a5c7d45321c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:58.836797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9dd32-94bc-4807-9ed7-a5c7d45321c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:58.841453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9dd32-94bc-4807-9ed7-a5c7d45321c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:58.841964 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9dd32-94bc-4807-9ed7-a5c7d45321c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:58.843598 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9dd32-94bc-4807-9ed7-a5c7d45321c3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:58.865839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 114/227] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:12:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:58.874148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9dd32-94bc-4807-9ed7-a5c7d45321c3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:58.989002 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9dd32-94bc-4807-9ed7-a5c7d45321c3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:59.069163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bb2ff-3ac5-4147-9284-0f79583715ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:59.070342 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bb2ff-3ac5-4147-9284-0f79583715ef None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:59.071843 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bb2ff-3ac5-4147-9284-0f79583715ef None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:59.072518 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bb2ff-3ac5-4147-9284-0f79583715ef None admin] PATH_INFO: `/v3/users/62bf7c2b073e4fe783f3ebeb3fa59fef` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:59.076079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4bb2ff-3ac5-4147-9284-0f79583715ef None admin] RBAC: Authorizing `identity:get_user(user_id=62bf7c2b073e4fe783f3ebeb3fa59fef)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:59.077971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4bb2ff-3ac5-4147-9284-0f79583715ef None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:59.080105 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 115/228] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:12:58 2022] GET /identity/v3/users/62bf7c2b073e4fe783f3ebeb3fa59fef => generated 283 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:12:59.088711 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bb2ff-3ac5-4147-9284-0f79583715ef None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:59.097275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8db1e9-01f8-41e7-b912-bc5f30222c7e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:59.098808 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:59.134196 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0cfd92-7273-4d62-9ed9-ee56257f7655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:59.134869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cfd92-7273-4d62-9ed9-ee56257f7655 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:59.135147 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cfd92-7273-4d62-9ed9-ee56257f7655 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:59.135416 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0cfd92-7273-4d62-9ed9-ee56257f7655 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:59.136175 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0cfd92-7273-4d62-9ed9-ee56257f7655 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:12:59.136480 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0cfd92-7273-4d62-9ed9-ee56257f7655 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:59.137861 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0cfd92-7273-4d62-9ed9-ee56257f7655 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:59.146555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:59.147350 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 114/229] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:12:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:59.154907 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0cfd92-7273-4d62-9ed9-ee56257f7655 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:59.274196 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bb2ff-3ac5-4147-9284-0f79583715ef None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:59.311408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0cfd92-7273-4d62-9ed9-ee56257f7655 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:59.354719 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddb315-fb9d-4548-80df-f5411f1da31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:59.355547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddb315-fb9d-4548-80df-f5411f1da31d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:59.355830 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddb315-fb9d-4548-80df-f5411f1da31d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:59.356100 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddb315-fb9d-4548-80df-f5411f1da31d None admin] PATH_INFO: `/v3/users/62bf7c2b073e4fe783f3ebeb3fa59fef` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:59.359650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddb315-fb9d-4548-80df-f5411f1da31d None admin] RBAC: Authorizing `identity:get_user(user_id=62bf7c2b073e4fe783f3ebeb3fa59fef)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:59.361535 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddb315-fb9d-4548-80df-f5411f1da31d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:59.365238 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 115/230] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:12:59 2022] GET /identity/v3/users/62bf7c2b073e4fe783f3ebeb3fa59fef => generated 283 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:59.371084 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7685c9-0e24-4518-8977-3437e179e135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:59.371994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7685c9-0e24-4518-8977-3437e179e135 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:59.372270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7685c9-0e24-4518-8977-3437e179e135 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:59.372539 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7685c9-0e24-4518-8977-3437e179e135 None admin] PATH_INFO: `/v3/projects/195eb382a49146b6b27a5cdc500c1a05` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:59.373948 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddb315-fb9d-4548-80df-f5411f1da31d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:59.374928 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7685c9-0e24-4518-8977-3437e179e135 None admin] RBAC: Authorizing `identity:get_project(project_id=195eb382a49146b6b27a5cdc500c1a05)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:59.376442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7685c9-0e24-4518-8977-3437e179e135 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:59.386379 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 116/231] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:12:59 2022] GET /identity/v3/projects/195eb382a49146b6b27a5cdc500c1a05 => generated 302 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:59.391718 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7685c9-0e24-4518-8977-3437e179e135 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:59.462991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:59.507393 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7685c9-0e24-4518-8977-3437e179e135 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:59.511288 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:12:59.544796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1674d-482d-4637-a911-f9fdb914a3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:59.550684 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1674d-482d-4637-a911-f9fdb914a3fe None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:59.550684 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1674d-482d-4637-a911-f9fdb914a3fe None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:59.550684 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1674d-482d-4637-a911-f9fdb914a3fe None admin] PATH_INFO: `/v3/projects/195eb382a49146b6b27a5cdc500c1a05/users/62bf7c2b073e4fe783f3ebeb3fa59fef/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:59.550684 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1674d-482d-4637-a911-f9fdb914a3fe None admin] RBAC: Authorizing `identity:create_grant(project_id=195eb382a49146b6b27a5cdc500c1a05, user_id=62bf7c2b073e4fe783f3ebeb3fa59fef, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:59.550684 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1674d-482d-4637-a911-f9fdb914a3fe None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:59.607438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddb315-fb9d-4548-80df-f5411f1da31d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:59.641654 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 117/232] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 30 16:12:59 2022] PUT /identity/v3/projects/195eb382a49146b6b27a5cdc500c1a05/users/62bf7c2b073e4fe783f3ebeb3fa59fef/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:12:59.649627 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8bb6c-0d6a-4fb5-9082-31c24d60bfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:59.650769 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8bb6c-0d6a-4fb5-9082-31c24d60bfac None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:59.651671 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8bb6c-0d6a-4fb5-9082-31c24d60bfac None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:59.651671 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8bb6c-0d6a-4fb5-9082-31c24d60bfac None admin] PATH_INFO: `/v3/projects/195eb382a49146b6b27a5cdc500c1a05` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:59.656526 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8bb6c-0d6a-4fb5-9082-31c24d60bfac None admin] RBAC: Authorizing `identity:get_project(project_id=195eb382a49146b6b27a5cdc500c1a05)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:59.658038 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8bb6c-0d6a-4fb5-9082-31c24d60bfac None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:59.659956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 116/233] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:12:59 2022] GET /identity/v3/projects/195eb382a49146b6b27a5cdc500c1a05 => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:12:59.666383 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1674d-482d-4637-a911-f9fdb914a3fe None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:12:59.840871 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1674d-482d-4637-a911-f9fdb914a3fe None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:12:59.882997 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10358e9f-0b22-4a28-a36d-b0e54cb073a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:12:59.885032 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10358e9f-0b22-4a28-a36d-b0e54cb073a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:12:59.885313 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10358e9f-0b22-4a28-a36d-b0e54cb073a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:12:59.885602 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10358e9f-0b22-4a28-a36d-b0e54cb073a5 None admin] PATH_INFO: `/v3/projects/195eb382a49146b6b27a5cdc500c1a05/users/62bf7c2b073e4fe783f3ebeb3fa59fef/roles/cfeec012f4ec48138c0e637e9a7fde61` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:12:59.887852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10358e9f-0b22-4a28-a36d-b0e54cb073a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=195eb382a49146b6b27a5cdc500c1a05, user_id=62bf7c2b073e4fe783f3ebeb3fa59fef, role_id=cfeec012f4ec48138c0e637e9a7fde61)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:12:59.889301 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10358e9f-0b22-4a28-a36d-b0e54cb073a5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:12:59.970081 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 118/234] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 30 16:12:59 2022] PUT /identity/v3/projects/195eb382a49146b6b27a5cdc500c1a05/users/62bf7c2b073e4fe783f3ebeb3fa59fef/roles/cfeec012f4ec48138c0e637e9a7fde61 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:13:00.496117 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e868e-27ec-4445-8863-b390c72e516a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:00.496684 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e868e-27ec-4445-8863-b390c72e516a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:00.496684 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e868e-27ec-4445-8863-b390c72e516a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:00.498515 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 117/235] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:00.513147 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a7c91-a53e-43fd-877f-6f13edffd011 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:00.513655 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a7c91-a53e-43fd-877f-6f13edffd011 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:00.513732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a7c91-a53e-43fd-877f-6f13edffd011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:00.540238 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-667a7c91-a53e-43fd-877f-6f13edffd011 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:00.541559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-667a7c91-a53e-43fd-877f-6f13edffd011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:00.561044 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:00.635519 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 119/236] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:13:00.654525 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21973748-d9a3-42b1-9f24-6eb47c64588c None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:00.654814 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21973748-d9a3-42b1-9f24-6eb47c64588c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:00.655100 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21973748-d9a3-42b1-9f24-6eb47c64588c None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:00.656780 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 118/237] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:00.661941 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f895f6-c095-4e53-a09e-3147c9c7262b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:00.662239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f895f6-c095-4e53-a09e-3147c9c7262b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:00.662524 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f895f6-c095-4e53-a09e-3147c9c7262b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:00.669023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21973748-d9a3-42b1-9f24-6eb47c64588c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:00.680758 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-22f895f6-c095-4e53-a09e-3147c9c7262b None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:00.681965 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-22f895f6-c095-4e53-a09e-3147c9c7262b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:00.759555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 120/238] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:00.785273 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f895f6-c095-4e53-a09e-3147c9c7262b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:00.835828 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:00.849137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21973748-d9a3-42b1-9f24-6eb47c64588c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:00.869429 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:00.892792 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f895f6-c095-4e53-a09e-3147c9c7262b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:00.898309 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d2c34-6191-4a68-b043-3f9f257be405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:00.899670 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d2c34-6191-4a68-b043-3f9f257be405 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:00.900142 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d2c34-6191-4a68-b043-3f9f257be405 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:00.900691 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d2c34-6191-4a68-b043-3f9f257be405 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:00.918831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d2c34-6191-4a68-b043-3f9f257be405 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:00.929379 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-532d2c34-6191-4a68-b043-3f9f257be405 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 16:13:00.929379 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:00.929379 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:00.929379 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:00.929379 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:00.929379 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:00.929379 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:00.929379 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:00.929379 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:00.929379 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:00.929379 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:00.929379 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:00.929379 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:00.929379 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:00.929379 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:00.929379 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:00.930841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:00.930841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:00.930841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:00.930841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:00.930841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:00.930841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:00.930841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:00.930841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:00.930841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:00.930841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:00.930841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:00.930841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:00.930841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:00.930841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:00.930841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:00.930841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:00.931682 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:00.931682 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:00.931682 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:00.931682 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:00.931682 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:00.931682 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:00.931682 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:00.931682 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:00.931682 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:00.931682 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:00.931682 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 16:13:00.931682 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:00.931682 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 119/239] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:13:00.932838 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89751d-4138-4a2e-953b-61338b21e57c None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:00.933020 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89751d-4138-4a2e-953b-61338b21e57c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:00.933171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafacf94-514e-4173-a205-5791cefc832b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:00.933336 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89751d-4138-4a2e-953b-61338b21e57c None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:00.938010 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafacf94-514e-4173-a205-5791cefc832b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:00.938010 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafacf94-514e-4173-a205-5791cefc832b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:00.938010 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafacf94-514e-4173-a205-5791cefc832b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:00.938508 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 120/240] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:00.939569 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafacf94-514e-4173-a205-5791cefc832b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:00.940386 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89751d-4138-4a2e-953b-61338b21e57c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:00.941051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafacf94-514e-4173-a205-5791cefc832b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:00.943001 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 121/241] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:00.944264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdcdf55-3666-4d1f-9e19-f4b448fb3b0f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:00.944551 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdcdf55-3666-4d1f-9e19-f4b448fb3b0f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:00.944830 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdcdf55-3666-4d1f-9e19-f4b448fb3b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:00.962765 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0bdcdf55-3666-4d1f-9e19-f4b448fb3b0f None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:00.964175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0bdcdf55-3666-4d1f-9e19-f4b448fb3b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:01.054658 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 122/242] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:01.056112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdcdf55-3666-4d1f-9e19-f4b448fb3b0f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:01.096772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89751d-4138-4a2e-953b-61338b21e57c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:01.132550 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e985d04-5461-45ac-bebc-787852b0a100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:01.133946 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e985d04-5461-45ac-bebc-787852b0a100 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:01.134327 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e985d04-5461-45ac-bebc-787852b0a100 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:01.134702 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e985d04-5461-45ac-bebc-787852b0a100 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:01.135587 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e985d04-5461-45ac-bebc-787852b0a100 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:01.135995 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e985d04-5461-45ac-bebc-787852b0a100 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:01.137473 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e985d04-5461-45ac-bebc-787852b0a100 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:01.138612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:01.147129 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 121/243] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:13:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:01.151750 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e985d04-5461-45ac-bebc-787852b0a100 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:01.166845 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdcdf55-3666-4d1f-9e19-f4b448fb3b0f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:01.174623 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:01.203745 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbcb25b-6cbb-44ef-a446-7bb7e1389842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:01.204426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbcb25b-6cbb-44ef-a446-7bb7e1389842 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:01.204710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbcb25b-6cbb-44ef-a446-7bb7e1389842 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:01.204994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbcb25b-6cbb-44ef-a446-7bb7e1389842 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:01.205749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbcb25b-6cbb-44ef-a446-7bb7e1389842 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:01.207276 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbcb25b-6cbb-44ef-a446-7bb7e1389842 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:01.257220 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e985d04-5461-45ac-bebc-787852b0a100 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:01.298317 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3439f5-a739-49d4-9ee5-5d7eb84389fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:01.299248 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3439f5-a739-49d4-9ee5-5d7eb84389fb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:01.299559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3439f5-a739-49d4-9ee5-5d7eb84389fb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:01.299837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3439f5-a739-49d4-9ee5-5d7eb84389fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:01.311283 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3439f5-a739-49d4-9ee5-5d7eb84389fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:01.319137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9d3439f5-a739-49d4-9ee5-5d7eb84389fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:13:01.319137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:01.319137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:01.319137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:01.319137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:01.319137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:01.319137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:01.319137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:01.319137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:01.319137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:01.319137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:01.319137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:01.319137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:01.319137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:01.319137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:01.319137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:01.321710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:01.321710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:01.321710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:01.321710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:01.321710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:01.321710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:01.321710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:01.321710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:01.321710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:01.321710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:01.321710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:01.321710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:01.321710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:01.321710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:01.321710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:01.321710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:01.322606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:01.322606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:01.322606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:01.322606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:01.322606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:01.322606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:01.322606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:01.322606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:01.322606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:01.322606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:01.322606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:13:01.322606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:01.326699 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 122/244] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:01.327332 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3439f5-a739-49d4-9ee5-5d7eb84389fb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:01.329397 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 123/245] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 30 16:13:01 2022] POST /identity/v3/users => generated 293 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:13:01.336720 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbcb25b-6cbb-44ef-a446-7bb7e1389842 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:01.500218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbcb25b-6cbb-44ef-a446-7bb7e1389842 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:01.529695 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:01.549533 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3439f5-a739-49d4-9ee5-5d7eb84389fb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:01.567912 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:01.581041 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfee639-6fb2-4cd0-85fc-bae2937e21e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:01.581713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfee639-6fb2-4cd0-85fc-bae2937e21e3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:01.581989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfee639-6fb2-4cd0-85fc-bae2937e21e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:01.582260 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfee639-6fb2-4cd0-85fc-bae2937e21e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:01.584505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfee639-6fb2-4cd0-85fc-bae2937e21e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:01.584505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfee639-6fb2-4cd0-85fc-bae2937e21e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:01.584753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfee639-6fb2-4cd0-85fc-bae2937e21e3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:01.596189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fbfa0c-825d-463f-a249-8a7b4368daf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:01.597208 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 124/246] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:13:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:01.597730 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbfa0c-825d-463f-a249-8a7b4368daf4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:01.598188 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbfa0c-825d-463f-a249-8a7b4368daf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:01.598638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fbfa0c-825d-463f-a249-8a7b4368daf4 None admin] PATH_INFO: `/v3/users/eacedf5ae1aa44e1aa6dac0ebedf4b57` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:01.603276 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfee639-6fb2-4cd0-85fc-bae2937e21e3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:01.614818 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fbfa0c-825d-463f-a249-8a7b4368daf4 None admin] RBAC: Authorizing `identity:get_user(user_id=eacedf5ae1aa44e1aa6dac0ebedf4b57)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:01.616660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fbfa0c-825d-463f-a249-8a7b4368daf4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:01.619300 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 123/247] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:13:01 2022] GET /identity/v3/users/eacedf5ae1aa44e1aa6dac0ebedf4b57 => generated 297 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:01.625930 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fbfa0c-825d-463f-a249-8a7b4368daf4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:01.742143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfee639-6fb2-4cd0-85fc-bae2937e21e3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:01.759959 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fbfa0c-825d-463f-a249-8a7b4368daf4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:01.796425 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70484f1-477e-44fb-b6f2-a1a0ec980839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:01.797313 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70484f1-477e-44fb-b6f2-a1a0ec980839 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:01.797594 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70484f1-477e-44fb-b6f2-a1a0ec980839 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:01.797869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70484f1-477e-44fb-b6f2-a1a0ec980839 None admin] PATH_INFO: `/v3/users/0f9c97c3ab734e1f9f553ccd41e81008` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:01.799246 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70484f1-477e-44fb-b6f2-a1a0ec980839 None admin] RBAC: Authorizing `identity:get_user(user_id=0f9c97c3ab734e1f9f553ccd41e81008)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:01.800896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70484f1-477e-44fb-b6f2-a1a0ec980839 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:01.802933 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 125/248] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:13:01 2022] GET /identity/v3/users/0f9c97c3ab734e1f9f553ccd41e81008 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:01.809324 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ad696-d003-48a8-97d9-76fd20c87117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:01.809495 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70484f1-477e-44fb-b6f2-a1a0ec980839 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:01.810423 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ad696-d003-48a8-97d9-76fd20c87117 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:01.810851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ad696-d003-48a8-97d9-76fd20c87117 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:01.811269 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436ad696-d003-48a8-97d9-76fd20c87117 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:01.830918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:01.895364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:01.914842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436ad696-d003-48a8-97d9-76fd20c87117 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:01.916575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436ad696-d003-48a8-97d9-76fd20c87117 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:01.958458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70484f1-477e-44fb-b6f2-a1a0ec980839 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:02.001379 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa66ad5-a2b0-4570-a905-44b852b5a2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:02.002305 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa66ad5-a2b0-4570-a905-44b852b5a2ef None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:02.002592 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa66ad5-a2b0-4570-a905-44b852b5a2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:02.002873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa66ad5-a2b0-4570-a905-44b852b5a2ef None admin] PATH_INFO: `/v3/projects/195eb382a49146b6b27a5cdc500c1a05` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:02.005159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa66ad5-a2b0-4570-a905-44b852b5a2ef None admin] RBAC: Authorizing `identity:get_project(project_id=195eb382a49146b6b27a5cdc500c1a05)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:02.006480 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa66ad5-a2b0-4570-a905-44b852b5a2ef None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:02.008804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 126/249] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:13:01 2022] GET /identity/v3/projects/195eb382a49146b6b27a5cdc500c1a05 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:02.016274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa66ad5-a2b0-4570-a905-44b852b5a2ef None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:02.063033 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 124/250] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 16:13:02.068299 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ad696-d003-48a8-97d9-76fd20c87117 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:02.120203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:02.190415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436ad696-d003-48a8-97d9-76fd20c87117 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:02.207571 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa66ad5-a2b0-4570-a905-44b852b5a2ef None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:02.231971 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa52cd-789d-4a2a-80ae-4c5b55bf5b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:02.233361 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa52cd-789d-4a2a-80ae-4c5b55bf5b2d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:02.233840 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa52cd-789d-4a2a-80ae-4c5b55bf5b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:02.234306 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fa52cd-789d-4a2a-80ae-4c5b55bf5b2d None admin] PATH_INFO: `/v3/projects/195eb382a49146b6b27a5cdc500c1a05` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:02.235716 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fa52cd-789d-4a2a-80ae-4c5b55bf5b2d None admin] RBAC: Authorizing `identity:get_project(project_id=195eb382a49146b6b27a5cdc500c1a05)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:02.237550 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fa52cd-789d-4a2a-80ae-4c5b55bf5b2d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:02.239588 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 125/251] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:13:02 2022] GET /identity/v3/projects/195eb382a49146b6b27a5cdc500c1a05 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:02.246329 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa52cd-789d-4a2a-80ae-4c5b55bf5b2d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:02.250295 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:02.269064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d734ccc-0def-4bc2-954c-56d8d417e69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:02.270351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d734ccc-0def-4bc2-954c-56d8d417e69a None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:02.270896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d734ccc-0def-4bc2-954c-56d8d417e69a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:02.271383 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d734ccc-0def-4bc2-954c-56d8d417e69a None admin] PATH_INFO: `/v3/projects/195eb382a49146b6b27a5cdc500c1a05/users/0f9c97c3ab734e1f9f553ccd41e81008/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:02.277886 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d734ccc-0def-4bc2-954c-56d8d417e69a None admin] RBAC: Authorizing `identity:create_grant(project_id=195eb382a49146b6b27a5cdc500c1a05, user_id=0f9c97c3ab734e1f9f553ccd41e81008, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:02.280938 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d734ccc-0def-4bc2-954c-56d8d417e69a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:02.374891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 127/252] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 30 16:13:02 2022] PUT /identity/v3/projects/195eb382a49146b6b27a5cdc500c1a05/users/0f9c97c3ab734e1f9f553ccd41e81008/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:13:02.390132 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fa52cd-789d-4a2a-80ae-4c5b55bf5b2d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:02.403156 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:02.417313 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6709f-1d50-4921-9e0f-0c56001adc85 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:02.417649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6709f-1d50-4921-9e0f-0c56001adc85 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:02.417952 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6709f-1d50-4921-9e0f-0c56001adc85 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:02.419689 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 128/253] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:13:02.425147 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60299484-34c7-4334-a87a-de1653f444db None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:02.425441 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60299484-34c7-4334-a87a-de1653f444db None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:02.425725 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60299484-34c7-4334-a87a-de1653f444db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:02.433537 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3f1d9-4dbe-485e-b3e3-dde51b4edc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:02.434545 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3f1d9-4dbe-485e-b3e3-dde51b4edc78 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:02.435054 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3f1d9-4dbe-485e-b3e3-dde51b4edc78 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:02.435534 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3f1d9-4dbe-485e-b3e3-dde51b4edc78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:02.436588 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3f1d9-4dbe-485e-b3e3-dde51b4edc78 None admin] RBAC: Adding query filter params (role.id=6e5a95d194ca42cab7a30baff4554fd3, scope.project.id=195eb382a49146b6b27a5cdc500c1a05, user.id=eacedf5ae1aa44e1aa6dac0ebedf4b57) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:02.437106 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3f1d9-4dbe-485e-b3e3-dde51b4edc78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:02.438838 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3f1d9-4dbe-485e-b3e3-dde51b4edc78 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:02.445903 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 126/254] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:13:02 2022] GET /identity/v3/role_assignments?user.id=eacedf5ae1aa44e1aa6dac0ebedf4b57&scope.project.id=195eb382a49146b6b27a5cdc500c1a05&role.id=6e5a95d194ca42cab7a30baff4554fd3 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:02.446456 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-60299484-34c7-4334-a87a-de1653f444db None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:02.451509 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-60299484-34c7-4334-a87a-de1653f444db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:02.561176 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 129/255] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:02.604130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3f1d9-4dbe-485e-b3e3-dde51b4edc78 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:02.702956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab21bf1-7c08-48f6-8277-b5c13912cd1b None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:02.704114 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab21bf1-7c08-48f6-8277-b5c13912cd1b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:02.704603 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab21bf1-7c08-48f6-8277-b5c13912cd1b None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:02.706650 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 130/256] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:02.713306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8dad7-6e26-4e01-b0d0-8338e7d04697 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:02.713898 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8dad7-6e26-4e01-b0d0-8338e7d04697 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:02.714397 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8dad7-6e26-4e01-b0d0-8338e7d04697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:02.744730 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ffd8dad7-6e26-4e01-b0d0-8338e7d04697 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:02.746382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ffd8dad7-6e26-4e01-b0d0-8338e7d04697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:02.746969 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3f1d9-4dbe-485e-b3e3-dde51b4edc78 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:02.786091 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bcb2bf-7b79-4e2a-a55a-12f735bfb5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:02.786909 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcb2bf-7b79-4e2a-a55a-12f735bfb5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:02.787194 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcb2bf-7b79-4e2a-a55a-12f735bfb5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:02.787489 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bcb2bf-7b79-4e2a-a55a-12f735bfb5f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:02.798467 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bcb2bf-7b79-4e2a-a55a-12f735bfb5f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:02.806789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-11bcb2bf-7b79-4e2a-a55a-12f735bfb5f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:13:02.806789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:02.806789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:02.806789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:02.806789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:02.806789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:02.806789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:02.806789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:02.806789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:02.806789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:02.806789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:02.806789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:02.806789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:02.806789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:02.806789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:02.806789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:02.807758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:02.807758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:02.807758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:02.807758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:02.807758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:02.807758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:02.807758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:02.807758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:02.807758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:02.807758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:02.807758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:02.807758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:02.807758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:02.807758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:02.807758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:02.807758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:02.808656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:02.808656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:02.808656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:02.808656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:02.808656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:02.808656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:02.808656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:02.808656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:02.808656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:02.808656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:02.808656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:13:02.808656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:02.808656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 127/257] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:13:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:02.819299 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bcb2bf-7b79-4e2a-a55a-12f735bfb5f9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:02.849933 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 131/258] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:13:02.864579 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:02.876984 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd8dad7-6e26-4e01-b0d0-8338e7d04697 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:02.973602 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:02.997291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bcb2bf-7b79-4e2a-a55a-12f735bfb5f9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:03.064567 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173bfb14-6a3f-4623-9e2e-c1fef1fc93ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:03.065679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173bfb14-6a3f-4623-9e2e-c1fef1fc93ab None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:03.066147 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173bfb14-6a3f-4623-9e2e-c1fef1fc93ab None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:03.066607 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173bfb14-6a3f-4623-9e2e-c1fef1fc93ab None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:03.067976 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173bfb14-6a3f-4623-9e2e-c1fef1fc93ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:03.068500 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173bfb14-6a3f-4623-9e2e-c1fef1fc93ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:03.071373 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173bfb14-6a3f-4623-9e2e-c1fef1fc93ab None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:03.084854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 128/259] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:03.090555 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173bfb14-6a3f-4623-9e2e-c1fef1fc93ab None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:03.107361 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd8dad7-6e26-4e01-b0d0-8338e7d04697 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:03.170992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c3f46-48af-4b21-9066-b304b370f1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:03.171844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c3f46-48af-4b21-9066-b304b370f1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:03.172129 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c3f46-48af-4b21-9066-b304b370f1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:03.172406 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c3f46-48af-4b21-9066-b304b370f1d2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:03.192149 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c3f46-48af-4b21-9066-b304b370f1d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:03.208369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-141c3f46-48af-4b21-9066-b304b370f1d2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:13:03.208369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:03.208369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:03.208369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:03.208369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:03.208369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:03.208369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:03.208369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:03.208369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:03.208369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:03.208369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:03.208369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:03.208369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:03.208369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:03.208369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:03.208369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:03.209279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:03.209279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:03.209279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:03.209279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:03.209279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:03.209279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:03.209279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:03.209279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:03.209279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:03.209279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:03.209279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:03.209279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:03.209279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:03.209279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:03.209279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:03.209279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:03.210055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:03.210055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:03.210055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:03.210055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:03.210055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:03.210055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:03.210055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:03.210055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:03.210055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:03.210055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:03.210055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:13:03.210055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:03.213038 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 132/260] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 30 16:13:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:13:03.218936 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c3f46-48af-4b21-9066-b304b370f1d2 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:03.257290 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:03.267787 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173bfb14-6a3f-4623-9e2e-c1fef1fc93ab None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:03.347079 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236de31-2aad-4716-be76-c3969e148d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:03.348040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236de31-2aad-4716-be76-c3969e148d41 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:03.348324 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236de31-2aad-4716-be76-c3969e148d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:03.348601 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7236de31-2aad-4716-be76-c3969e148d41 None admin] PATH_INFO: `/v3/users/62bf7c2b073e4fe783f3ebeb3fa59fef` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:03.349975 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7236de31-2aad-4716-be76-c3969e148d41 None admin] RBAC: Authorizing `identity:get_user(user_id=62bf7c2b073e4fe783f3ebeb3fa59fef)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:03.352149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7236de31-2aad-4716-be76-c3969e148d41 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:03.354076 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 129/261] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:13:03 2022] GET /identity/v3/users/62bf7c2b073e4fe783f3ebeb3fa59fef => generated 283 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:03.359532 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236de31-2aad-4716-be76-c3969e148d41 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:03.457409 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c3f46-48af-4b21-9066-b304b370f1d2 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:03.473791 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:03.500002 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7236de31-2aad-4716-be76-c3969e148d41 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:03.535750 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42457cbe-034e-411a-b694-bf892854ad5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:03.535932 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe40de-17e7-4529-88ec-29fe46bc78b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:03.536358 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe40de-17e7-4529-88ec-29fe46bc78b5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:03.536649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe40de-17e7-4529-88ec-29fe46bc78b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:03.536930 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe40de-17e7-4529-88ec-29fe46bc78b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:03.537493 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42457cbe-034e-411a-b694-bf892854ad5c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:03.537692 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe40de-17e7-4529-88ec-29fe46bc78b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:03.538033 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe40de-17e7-4529-88ec-29fe46bc78b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:03.538373 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42457cbe-034e-411a-b694-bf892854ad5c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:03.538840 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42457cbe-034e-411a-b694-bf892854ad5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:03.540108 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe40de-17e7-4529-88ec-29fe46bc78b5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:03.561507 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 133/262] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:13:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:03.567351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe40de-17e7-4529-88ec-29fe46bc78b5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:03.609121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:03.632273 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42457cbe-034e-411a-b694-bf892854ad5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:03.634493 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42457cbe-034e-411a-b694-bf892854ad5c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:03.744304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 130/263] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:13:03.749569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42457cbe-034e-411a-b694-bf892854ad5c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:03.779181 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe40de-17e7-4529-88ec-29fe46bc78b5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:03.832923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273ce79-378a-40ac-98a9-94c24a5f0577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:03.833820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273ce79-378a-40ac-98a9-94c24a5f0577 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:03.834157 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273ce79-378a-40ac-98a9-94c24a5f0577 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:03.834441 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273ce79-378a-40ac-98a9-94c24a5f0577 None admin] PATH_INFO: `/v3/users/62bf7c2b073e4fe783f3ebeb3fa59fef` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:03.835715 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5273ce79-378a-40ac-98a9-94c24a5f0577 None admin] RBAC: Authorizing `identity:get_user(user_id=62bf7c2b073e4fe783f3ebeb3fa59fef)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:03.838044 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5273ce79-378a-40ac-98a9-94c24a5f0577 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:03.840065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 134/264] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:13:03 2022] GET /identity/v3/users/62bf7c2b073e4fe783f3ebeb3fa59fef => generated 283 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:03.843908 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:03.845541 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273ce79-378a-40ac-98a9-94c24a5f0577 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:03.860817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42457cbe-034e-411a-b694-bf892854ad5c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:03.887685 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:03.900184 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300e212-7515-4607-9ca5-5c362d4973cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:03.901639 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300e212-7515-4607-9ca5-5c362d4973cd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:03.902121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300e212-7515-4607-9ca5-5c362d4973cd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:03.902614 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300e212-7515-4607-9ca5-5c362d4973cd None admin] PATH_INFO: `/v3/projects/195eb382a49146b6b27a5cdc500c1a05` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:03.904098 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300e212-7515-4607-9ca5-5c362d4973cd None admin] RBAC: Authorizing `identity:get_project(project_id=195eb382a49146b6b27a5cdc500c1a05)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:03.905817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300e212-7515-4607-9ca5-5c362d4973cd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:03.908130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 131/265] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:13:03 2022] GET /identity/v3/projects/195eb382a49146b6b27a5cdc500c1a05 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:03.916425 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300e212-7515-4607-9ca5-5c362d4973cd None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:03.952864 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273ce79-378a-40ac-98a9-94c24a5f0577 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:03.989705 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a19e5a-f467-4418-90b2-523fad47e541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:03.990358 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a19e5a-f467-4418-90b2-523fad47e541 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:03.990640 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a19e5a-f467-4418-90b2-523fad47e541 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:03.990918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a19e5a-f467-4418-90b2-523fad47e541 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:04.062570 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a19e5a-f467-4418-90b2-523fad47e541 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:04.064084 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a19e5a-f467-4418-90b2-523fad47e541 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:04.083140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300e212-7515-4607-9ca5-5c362d4973cd None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:04.105601 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:04.116575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:04.123701 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9da6f9-1f1b-4848-bd0c-9a02b66db9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:04.124641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9da6f9-1f1b-4848-bd0c-9a02b66db9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:04.125141 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9da6f9-1f1b-4848-bd0c-9a02b66db9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:04.125615 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9da6f9-1f1b-4848-bd0c-9a02b66db9d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:04.126637 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9da6f9-1f1b-4848-bd0c-9a02b66db9d3 None admin] RBAC: Adding query filter params (role.id=8acc34a163874a60b03728b814eaf075, scope.project.id=195eb382a49146b6b27a5cdc500c1a05, user.id=62bf7c2b073e4fe783f3ebeb3fa59fef) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:04.127149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9da6f9-1f1b-4848-bd0c-9a02b66db9d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:04.128871 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9da6f9-1f1b-4848-bd0c-9a02b66db9d3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:04.135430 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 132/266] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:13:03 2022] GET /identity/v3/role_assignments?user.id=62bf7c2b073e4fe783f3ebeb3fa59fef&scope.project.id=195eb382a49146b6b27a5cdc500c1a05&role.id=8acc34a163874a60b03728b814eaf075 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:04.173678 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 135/267] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 16:13:04.179323 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9da6f9-1f1b-4848-bd0c-9a02b66db9d3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:04.309859 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9da6f9-1f1b-4848-bd0c-9a02b66db9d3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:04.352512 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c702d-0e21-4a83-9b10-9870c41d5270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:04.353758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c702d-0e21-4a83-9b10-9870c41d5270 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:04.354242 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c702d-0e21-4a83-9b10-9870c41d5270 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:04.354713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c702d-0e21-4a83-9b10-9870c41d5270 None admin] PATH_INFO: `/v3/projects/195eb382a49146b6b27a5cdc500c1a05` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:04.360352 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c702d-0e21-4a83-9b10-9870c41d5270 None admin] RBAC: Authorizing `identity:get_project(project_id=195eb382a49146b6b27a5cdc500c1a05)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:04.362252 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118c702d-0e21-4a83-9b10-9870c41d5270 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:04.364364 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 133/268] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:13:04 2022] GET /identity/v3/projects/195eb382a49146b6b27a5cdc500c1a05 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:04.376751 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a19e5a-f467-4418-90b2-523fad47e541 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:04.493867 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a19e5a-f467-4418-90b2-523fad47e541 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:04.530497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc6b335-b696-4307-b779-99a024852055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:04.531165 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6b335-b696-4307-b779-99a024852055 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:04.531451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6b335-b696-4307-b779-99a024852055 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:04.531761 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6b335-b696-4307-b779-99a024852055 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:04.532623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6b335-b696-4307-b779-99a024852055 None admin] RBAC: Adding query filter params (role.id=cfeec012f4ec48138c0e637e9a7fde61, scope.project.id=195eb382a49146b6b27a5cdc500c1a05, user.id=62bf7c2b073e4fe783f3ebeb3fa59fef) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:04.532940 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6b335-b696-4307-b779-99a024852055 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:04.534443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc6b335-b696-4307-b779-99a024852055 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:04.541160 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 136/269] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:13:04 2022] GET /identity/v3/role_assignments?user.id=62bf7c2b073e4fe783f3ebeb3fa59fef&scope.project.id=195eb382a49146b6b27a5cdc500c1a05&role.id=cfeec012f4ec48138c0e637e9a7fde61 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:04.973051 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2874398-64a9-44f3-8029-47183bb24d37 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:04.973587 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2874398-64a9-44f3-8029-47183bb24d37 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:04.973692 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2874398-64a9-44f3-8029-47183bb24d37 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:04.975334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 134/270] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:04.981599 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7bd4f4-32c5-4bdf-b766-d802b96f073f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:04.982055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7bd4f4-32c5-4bdf-b766-d802b96f073f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:04.982237 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7bd4f4-32c5-4bdf-b766-d802b96f073f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:04.987368 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:05.002248 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-be7bd4f4-32c5-4bdf-b766-d802b96f073f None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:05.003559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-be7bd4f4-32c5-4bdf-b766-d802b96f073f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:05.081511 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bef065-de58-434e-a77a-3c4568211494 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:05.081828 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bef065-de58-434e-a77a-3c4568211494 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:05.082099 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bef065-de58-434e-a77a-3c4568211494 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:05.083941 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 135/271] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:05.084695 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 137/272] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:05.089762 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4db46e-45ff-46bf-8716-02475be85b64 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:05.090058 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4db46e-45ff-46bf-8716-02475be85b64 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:05.090344 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4db46e-45ff-46bf-8716-02475be85b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:05.109289 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ee4db46e-45ff-46bf-8716-02475be85b64 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:05.110505 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ee4db46e-45ff-46bf-8716-02475be85b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:05.117737 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7bd4f4-32c5-4bdf-b766-d802b96f073f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:05.177407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:05.190120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 136/273] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:05.215858 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4db46e-45ff-46bf-8716-02475be85b64 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:05.239515 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7bd4f4-32c5-4bdf-b766-d802b96f073f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:05.279842 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:05.295783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66838d-bb4e-4d13-a6ef-1cfe74bcc789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:05.301067 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66838d-bb4e-4d13-a6ef-1cfe74bcc789 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:05.301487 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66838d-bb4e-4d13-a6ef-1cfe74bcc789 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:05.301894 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66838d-bb4e-4d13-a6ef-1cfe74bcc789 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:05.317896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e66838d-bb4e-4d13-a6ef-1cfe74bcc789 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:05.330504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-6e66838d-bb4e-4d13-a6ef-1cfe74bcc789 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:13:05.330504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:05.330504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:05.330504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:05.330504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:05.330504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:05.330504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:05.330504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:05.330504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:05.330504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:05.330504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:05.330504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:05.330504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:05.330504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:05.330504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:05.330504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:05.332082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:05.332082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:05.332082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:05.332082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:05.332082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:05.332082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:05.332082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:05.332082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:05.332082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:05.332082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:05.332082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:05.332082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:05.332082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:05.332082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:05.332082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:05.332082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:05.333223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:05.333223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:05.333223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:05.333223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:05.333223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:05.333223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:05.333223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:05.333223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:05.333223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:05.333223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:05.333223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:13:05.333223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:05.334542 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 138/274] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:05.336863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4db46e-45ff-46bf-8716-02475be85b64 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:05.341898 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66838d-bb4e-4d13-a6ef-1cfe74bcc789 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:05.389540 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7cbe3d-eadd-485f-8549-c67f9a8de573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:05.390498 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7cbe3d-eadd-485f-8549-c67f9a8de573 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:05.390855 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7cbe3d-eadd-485f-8549-c67f9a8de573 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:05.391214 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7cbe3d-eadd-485f-8549-c67f9a8de573 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:05.392646 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7cbe3d-eadd-485f-8549-c67f9a8de573 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:05.394277 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7cbe3d-eadd-485f-8549-c67f9a8de573 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:05.396776 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 137/275] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:05.402695 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7cbe3d-eadd-485f-8549-c67f9a8de573 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:05.465651 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66838d-bb4e-4d13-a6ef-1cfe74bcc789 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:05.467080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:05.504701 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358d390-c69d-4d14-8ed8-4ea14a605942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:05.505335 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358d390-c69d-4d14-8ed8-4ea14a605942 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:05.505619 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358d390-c69d-4d14-8ed8-4ea14a605942 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:05.505899 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358d390-c69d-4d14-8ed8-4ea14a605942 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:05.506651 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358d390-c69d-4d14-8ed8-4ea14a605942 None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:05.507016 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358d390-c69d-4d14-8ed8-4ea14a605942 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:05.508986 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4358d390-c69d-4d14-8ed8-4ea14a605942 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:05.514096 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7cbe3d-eadd-485f-8549-c67f9a8de573 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:05.519252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 139/276] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:13:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:05.521131 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b60a4-75a3-447f-a685-fb8c4a20f970 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:05.521709 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b60a4-75a3-447f-a685-fb8c4a20f970 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:05.522204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b60a4-75a3-447f-a685-fb8c4a20f970 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:05.525773 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 140/277] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:05.526781 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b60a4-75a3-447f-a685-fb8c4a20f970 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:05.555932 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2fe8a5-d6f1-4adf-b51c-71d52005943f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:05.556625 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fe8a5-d6f1-4adf-b51c-71d52005943f None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:05.556909 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fe8a5-d6f1-4adf-b51c-71d52005943f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:05.557186 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fe8a5-d6f1-4adf-b51c-71d52005943f None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:05.557911 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2fe8a5-d6f1-4adf-b51c-71d52005943f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:05.559359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2fe8a5-d6f1-4adf-b51c-71d52005943f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:05.593856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:05.638393 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b60a4-75a3-447f-a685-fb8c4a20f970 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:05.660718 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 138/278] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 30 16:13:05 2022] POST /identity/v3/users => generated 293 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:13:05.662271 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0e154-206c-46cb-80a0-882428ba4b6a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:05.662567 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0e154-206c-46cb-80a0-882428ba4b6a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:05.662849 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0e154-206c-46cb-80a0-882428ba4b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:05.681943 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768296b-7efc-4615-999c-25f3bd13c994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:05.683409 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768296b-7efc-4615-999c-25f3bd13c994 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:05.683824 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-25d0e154-206c-46cb-80a0-882428ba4b6a None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:05.684039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768296b-7efc-4615-999c-25f3bd13c994 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:05.684529 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768296b-7efc-4615-999c-25f3bd13c994 None admin] PATH_INFO: `/v3/users/baee4e1d8ef14cc7871ee186f960e301` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:05.685062 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-25d0e154-206c-46cb-80a0-882428ba4b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:05.704006 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768296b-7efc-4615-999c-25f3bd13c994 None admin] RBAC: Authorizing `identity:get_user(user_id=baee4e1d8ef14cc7871ee186f960e301)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:05.705506 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768296b-7efc-4615-999c-25f3bd13c994 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:05.707292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 141/279] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:13:05 2022] GET /identity/v3/users/baee4e1d8ef14cc7871ee186f960e301 => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:05.708611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fe060-7511-4a07-a69f-de908a538f5b None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:05.708902 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fe060-7511-4a07-a69f-de908a538f5b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:05.709183 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fe060-7511-4a07-a69f-de908a538f5b None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:05.710618 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 142/280] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:05.719521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fe060-7511-4a07-a69f-de908a538f5b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:05.775009 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 139/281] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:05.775009 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5522f60-69d7-42d2-a920-69bb1ffe1bff None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:05.775009 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5522f60-69d7-42d2-a920-69bb1ffe1bff None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:05.775009 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5522f60-69d7-42d2-a920-69bb1ffe1bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:05.802145 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f5522f60-69d7-42d2-a920-69bb1ffe1bff None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:05.803403 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f5522f60-69d7-42d2-a920-69bb1ffe1bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:05.839708 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62fe060-7511-4a07-a69f-de908a538f5b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:05.854788 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:05.857354 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:05.880440 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6249b-3218-41a1-8f9c-a7b7f18886ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:05.881391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6249b-3218-41a1-8f9c-a7b7f18886ed None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:05.881507 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 140/282] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:05.881585 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6249b-3218-41a1-8f9c-a7b7f18886ed None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:05.881652 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6249b-3218-41a1-8f9c-a7b7f18886ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:05.882505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6249b-3218-41a1-8f9c-a7b7f18886ed None admin] RBAC: Adding query filter params (role.id=9ab3e302ead54950b7c30103f1e7fda7, user.id=baee4e1d8ef14cc7871ee186f960e301, scope.system=all) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:05.882866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6249b-3218-41a1-8f9c-a7b7f18886ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:05.883844 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5522f60-69d7-42d2-a920-69bb1ffe1bff None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:05.884482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6249b-3218-41a1-8f9c-a7b7f18886ed None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:05.890195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 143/283] 10.222.0.67 () {56 vars in 1308 bytes} [Thu Jun 30 16:13:05 2022] GET /identity/v3/role_assignments?user.id=baee4e1d8ef14cc7871ee186f960e301&scope.system=all&role.id=9ab3e302ead54950b7c30103f1e7fda7 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:05.905984 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6249b-3218-41a1-8f9c-a7b7f18886ed None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:06.034114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5522f60-69d7-42d2-a920-69bb1ffe1bff None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:06.052545 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6249b-3218-41a1-8f9c-a7b7f18886ed None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:06.080227 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a08c91-e5cb-427a-ac5f-1b6181037049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:06.081750 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a08c91-e5cb-427a-ac5f-1b6181037049 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:06.082052 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a08c91-e5cb-427a-ac5f-1b6181037049 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:06.083338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a08c91-e5cb-427a-ac5f-1b6181037049 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:06.095410 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c5ff2-f9d1-4778-8d37-c1b553bc0433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:06.096243 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c5ff2-f9d1-4778-8d37-c1b553bc0433 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:06.096590 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c5ff2-f9d1-4778-8d37-c1b553bc0433 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:06.096877 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c5ff2-f9d1-4778-8d37-c1b553bc0433 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:06.098022 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a08c91-e5cb-427a-ac5f-1b6181037049 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:06.108913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-51a08c91-e5cb-427a-ac5f-1b6181037049 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:13:06.108913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:06.108913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:06.108913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:06.108913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:06.108913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:06.108913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:06.108913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:06.108913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:06.108913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:06.108913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:06.108913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:06.108913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:06.108913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:06.108913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:06.108913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:06.109837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:06.109837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:06.109837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:06.109837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:06.109837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:06.109837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:06.109837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:06.109837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:06.109837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:06.109837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:06.109837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:06.109837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:06.109837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:06.109837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:06.109837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:06.109837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:06.110619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:06.110619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:06.110619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:06.110619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:06.110619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:06.110619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:06.110619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:06.110619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:06.110619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:06.110619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:06.110619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:13:06.110619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:06.111238 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c5ff2-f9d1-4778-8d37-c1b553bc0433 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:06.111311 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 141/284] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:06.121286 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a08c91-e5cb-427a-ac5f-1b6181037049 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:06.122290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-883c5ff2-f9d1-4778-8d37-c1b553bc0433 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 16:13:06.122290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:06.122290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:06.122290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:06.122290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:06.122290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:06.122290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:06.122290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:06.122290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:06.122290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:06.122290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:06.122290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:06.122290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:06.122290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:06.122290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:06.122290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:06.124377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:06.124377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:06.124377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:06.124377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:06.124377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:06.124377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:06.124377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:06.124377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:06.124377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:06.124377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:06.124377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:06.124377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:06.124377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:06.124377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:06.124377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:06.124377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:06.125697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:06.125697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:06.125697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:06.125697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:06.125697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:06.125697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:06.125697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:06.125697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:06.125697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:06.125697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:06.125697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 16:13:06.125697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:06.125697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 144/285] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:06.130429 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c5ff2-f9d1-4778-8d37-c1b553bc0433 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:06.186303 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:06.187907 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:06.301178 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c5ff2-f9d1-4778-8d37-c1b553bc0433 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:06.313677 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a08c91-e5cb-427a-ac5f-1b6181037049 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:06.370985 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e040af-fd81-4f02-ada5-59ed6bed1ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:06.371709 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e040af-fd81-4f02-ada5-59ed6bed1ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:06.372070 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e040af-fd81-4f02-ada5-59ed6bed1ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:06.372353 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e040af-fd81-4f02-ada5-59ed6bed1ff9 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:06.373132 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e040af-fd81-4f02-ada5-59ed6bed1ff9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:06.373443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e040af-fd81-4f02-ada5-59ed6bed1ff9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:06.374018 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df1bf9-f893-4543-9661-bf14ef94701c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:06.374840 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e040af-fd81-4f02-ada5-59ed6bed1ff9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:06.375117 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df1bf9-f893-4543-9661-bf14ef94701c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:06.375629 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df1bf9-f893-4543-9661-bf14ef94701c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:06.376237 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df1bf9-f893-4543-9661-bf14ef94701c None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:06.377299 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df1bf9-f893-4543-9661-bf14ef94701c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:06.377810 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df1bf9-f893-4543-9661-bf14ef94701c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:06.379413 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df1bf9-f893-4543-9661-bf14ef94701c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:06.384766 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 142/286] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:13:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:06.390327 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e040af-fd81-4f02-ada5-59ed6bed1ff9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:06.396878 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 145/287] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:13:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:06.402955 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df1bf9-f893-4543-9661-bf14ef94701c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:06.552572 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:06.557232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:06.560128 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df1bf9-f893-4543-9661-bf14ef94701c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:06.561026 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e040af-fd81-4f02-ada5-59ed6bed1ff9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:06.625108 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5dc65-a3ad-463c-81bf-9ec2ddc7eb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:06.626374 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5dc65-a3ad-463c-81bf-9ec2ddc7eb2f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:06.627255 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5dc65-a3ad-463c-81bf-9ec2ddc7eb2f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:06.628194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5dc65-a3ad-463c-81bf-9ec2ddc7eb2f None admin] PATH_INFO: `/v3/users/eacedf5ae1aa44e1aa6dac0ebedf4b57` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:06.629889 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c5dc65-a3ad-463c-81bf-9ec2ddc7eb2f None admin] RBAC: Authorizing `identity:get_user(user_id=eacedf5ae1aa44e1aa6dac0ebedf4b57)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:06.633400 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a79f2e-ec50-484d-8af2-941af5053c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:06.635176 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c5dc65-a3ad-463c-81bf-9ec2ddc7eb2f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:06.637214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 146/288] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:13:06 2022] GET /identity/v3/users/eacedf5ae1aa44e1aa6dac0ebedf4b57 => generated 297 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:06.637818 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a79f2e-ec50-484d-8af2-941af5053c1d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:06.638341 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a79f2e-ec50-484d-8af2-941af5053c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:06.638818 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a79f2e-ec50-484d-8af2-941af5053c1d None admin] PATH_INFO: `/v3/users/0f9c97c3ab734e1f9f553ccd41e81008` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:06.640321 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a79f2e-ec50-484d-8af2-941af5053c1d None admin] RBAC: Authorizing `identity:get_user(user_id=0f9c97c3ab734e1f9f553ccd41e81008)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:06.641982 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a79f2e-ec50-484d-8af2-941af5053c1d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:06.643003 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5dc65-a3ad-463c-81bf-9ec2ddc7eb2f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:06.645241 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 143/289] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:13:06 2022] GET /identity/v3/users/0f9c97c3ab734e1f9f553ccd41e81008 => generated 297 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:06.655230 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a79f2e-ec50-484d-8af2-941af5053c1d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:06.777504 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a79f2e-ec50-484d-8af2-941af5053c1d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:06.816815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e10ab-3f7c-4e0f-833a-a2d46d311add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:06.817878 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e10ab-3f7c-4e0f-833a-a2d46d311add None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:06.818239 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e10ab-3f7c-4e0f-833a-a2d46d311add None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:06.818594 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e10ab-3f7c-4e0f-833a-a2d46d311add None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:06.847078 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5dc65-a3ad-463c-81bf-9ec2ddc7eb2f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:06.856599 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:06.892191 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e10ab-3f7c-4e0f-833a-a2d46d311add None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:06.894154 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e10ab-3f7c-4e0f-833a-a2d46d311add None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:06.926971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d51acc-95a2-4340-ac00-d064a72ce360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:06.928463 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d51acc-95a2-4340-ac00-d064a72ce360 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:06.929232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d51acc-95a2-4340-ac00-d064a72ce360 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:06.930529 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d51acc-95a2-4340-ac00-d064a72ce360 None admin] PATH_INFO: `/v3/projects/195eb382a49146b6b27a5cdc500c1a05` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:06.936651 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d51acc-95a2-4340-ac00-d064a72ce360 None admin] RBAC: Authorizing `identity:get_project(project_id=195eb382a49146b6b27a5cdc500c1a05)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:06.938421 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d51acc-95a2-4340-ac00-d064a72ce360 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:06.944933 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 147/290] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:13:06 2022] GET /identity/v3/projects/195eb382a49146b6b27a5cdc500c1a05 => generated 302 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:06.946370 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d51acc-95a2-4340-ac00-d064a72ce360 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:07.000002 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 144/291] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:13:07.005709 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e10ab-3f7c-4e0f-833a-a2d46d311add None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:07.032971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:07.075912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:07.111865 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e10ab-3f7c-4e0f-833a-a2d46d311add None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:07.148221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d05ed-3c04-4682-a811-6331a36384e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:07.149271 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d05ed-3c04-4682-a811-6331a36384e1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:07.149671 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d05ed-3c04-4682-a811-6331a36384e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:07.150020 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d05ed-3c04-4682-a811-6331a36384e1 None admin] PATH_INFO: `/v3/projects/195eb382a49146b6b27a5cdc500c1a05` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:07.151730 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d05ed-3c04-4682-a811-6331a36384e1 None admin] RBAC: Authorizing `identity:get_project(project_id=195eb382a49146b6b27a5cdc500c1a05)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:07.153328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d05ed-3c04-4682-a811-6331a36384e1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:07.155438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 145/292] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:13:07 2022] GET /identity/v3/projects/195eb382a49146b6b27a5cdc500c1a05 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:07.161165 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d05ed-3c04-4682-a811-6331a36384e1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:07.197798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d51acc-95a2-4340-ac00-d064a72ce360 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:07.272206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d05ed-3c04-4682-a811-6331a36384e1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:07.298101 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb00b84-883d-4315-9e7d-a15b3df35e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:07.299078 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb00b84-883d-4315-9e7d-a15b3df35e4f None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:07.299360 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb00b84-883d-4315-9e7d-a15b3df35e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:07.299662 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb00b84-883d-4315-9e7d-a15b3df35e4f None admin] PATH_INFO: `/v3/projects/195eb382a49146b6b27a5cdc500c1a05/users/eacedf5ae1aa44e1aa6dac0ebedf4b57/roles/6e5a95d194ca42cab7a30baff4554fd3` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:07.308531 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb00b84-883d-4315-9e7d-a15b3df35e4f None admin] RBAC: Authorizing `identity:create_grant(project_id=195eb382a49146b6b27a5cdc500c1a05, user_id=eacedf5ae1aa44e1aa6dac0ebedf4b57, role_id=6e5a95d194ca42cab7a30baff4554fd3)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:07.309949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb00b84-883d-4315-9e7d-a15b3df35e4f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:07.311415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968ebe6-4b22-4ace-b0b8-82f536922c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:07.312115 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968ebe6-4b22-4ace-b0b8-82f536922c0e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:07.312399 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968ebe6-4b22-4ace-b0b8-82f536922c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:07.312683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968ebe6-4b22-4ace-b0b8-82f536922c0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:07.313524 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968ebe6-4b22-4ace-b0b8-82f536922c0e None admin] RBAC: Adding query filter params (role.id=9ab3e302ead54950b7c30103f1e7fda7, scope.project.id=195eb382a49146b6b27a5cdc500c1a05, user.id=0f9c97c3ab734e1f9f553ccd41e81008) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:07.313841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968ebe6-4b22-4ace-b0b8-82f536922c0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:07.315280 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968ebe6-4b22-4ace-b0b8-82f536922c0e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:07.319815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:07.329741 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 146/293] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:13:07 2022] GET /identity/v3/role_assignments?user.id=0f9c97c3ab734e1f9f553ccd41e81008&scope.project.id=195eb382a49146b6b27a5cdc500c1a05&role.id=9ab3e302ead54950b7c30103f1e7fda7 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:07.387569 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 148/294] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 30 16:13:06 2022] PUT /identity/v3/projects/195eb382a49146b6b27a5cdc500c1a05/users/eacedf5ae1aa44e1aa6dac0ebedf4b57/roles/6e5a95d194ca42cab7a30baff4554fd3 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:13:08.238329 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f689d7-7c37-4ab8-aefe-fa859d7e39b2 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:08.238864 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f689d7-7c37-4ab8-aefe-fa859d7e39b2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:08.238864 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f689d7-7c37-4ab8-aefe-fa859d7e39b2 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:08.240630 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 147/295] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:08.247548 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd09b12-24db-4df9-9eca-495db88deee2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:08.248027 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd09b12-24db-4df9-9eca-495db88deee2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:08.248103 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd09b12-24db-4df9-9eca-495db88deee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:08.266671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0cd09b12-24db-4df9-9eca-495db88deee2 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:08.267888 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0cd09b12-24db-4df9-9eca-495db88deee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:08.286756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61b1b2-f7bf-427c-8316-fb55a759d12a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:08.287035 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61b1b2-f7bf-427c-8316-fb55a759d12a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:08.287315 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c61b1b2-f7bf-427c-8316-fb55a759d12a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:08.289532 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 148/296] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:13:08.294063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42c8b1-61b9-4269-aaa7-8020f3988cdc None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:08.294349 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42c8b1-61b9-4269-aaa7-8020f3988cdc None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:08.294626 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42c8b1-61b9-4269-aaa7-8020f3988cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:08.312156 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7f42c8b1-61b9-4269-aaa7-8020f3988cdc None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:08.313358 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7f42c8b1-61b9-4269-aaa7-8020f3988cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:08.343810 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 149/297] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:08.353063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4dcb41-eaa4-4de6-919a-841b13cf0c2a None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:08.353623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4dcb41-eaa4-4de6-919a-841b13cf0c2a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:08.354130 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4dcb41-eaa4-4de6-919a-841b13cf0c2a None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:08.360236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 150/298] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:08 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:08.361689 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febea6f-a114-4261-b5f8-86b6be06f1f2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:08.362248 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febea6f-a114-4261-b5f8-86b6be06f1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:08.362856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4febea6f-a114-4261-b5f8-86b6be06f1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:08.380424 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-4febea6f-a114-4261-b5f8-86b6be06f1f2 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:08.381624 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4febea6f-a114-4261-b5f8-86b6be06f1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:08.384267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 149/299] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:13:08.388752 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42c8b1-61b9-4269-aaa7-8020f3988cdc None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:08.405859 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:08.460195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 151/300] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:08.462631 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4febea6f-a114-4261-b5f8-86b6be06f1f2 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:08.507068 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42c8b1-61b9-4269-aaa7-8020f3988cdc None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:08.535780 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:08.543890 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d4f50a-afd0-49d2-bffe-5c4c606c6098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:08.544700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4f50a-afd0-49d2-bffe-5c4c606c6098 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:08.544980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4f50a-afd0-49d2-bffe-5c4c606c6098 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:08.545254 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d4f50a-afd0-49d2-bffe-5c4c606c6098 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:08.555632 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d4f50a-afd0-49d2-bffe-5c4c606c6098 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:08.563337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-37d4f50a-afd0-49d2-bffe-5c4c606c6098 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 16:13:08.563337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:08.563337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:08.563337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:08.563337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:08.563337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:08.563337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:08.563337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:08.563337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:08.563337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:08.563337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:08.563337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:08.563337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:08.563337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:08.563337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:08.563337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:08.564275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:08.564275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:08.564275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:08.564275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:08.564275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:08.564275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:08.564275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:08.564275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:08.564275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:08.564275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:08.564275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:08.564275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:08.564275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:08.564275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:08.564275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:08.564275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:08.565065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:08.565065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:08.565065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:08.565065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:08.565065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:08.565065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:08.565065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:08.565065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:08.565065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:08.565065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:08.565065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 16:13:08.565065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:08.565065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 150/301] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:08.565715 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d4f50a-afd0-49d2-bffe-5c4c606c6098 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:08.578476 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4febea6f-a114-4261-b5f8-86b6be06f1f2 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:08.615393 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe4968-7e1b-4af9-a1ca-2f78bad01777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:08.616283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe4968-7e1b-4af9-a1ca-2f78bad01777 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:08.616575 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe4968-7e1b-4af9-a1ca-2f78bad01777 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:08.616880 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe4968-7e1b-4af9-a1ca-2f78bad01777 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:08.618632 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fe4968-7e1b-4af9-a1ca-2f78bad01777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:08.620143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fe4968-7e1b-4af9-a1ca-2f78bad01777 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:08.626292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 152/302] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:08.626292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe4968-7e1b-4af9-a1ca-2f78bad01777 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:08.655761 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:08.689141 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d4f50a-afd0-49d2-bffe-5c4c606c6098 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:08.725641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d672e-7e5c-4d41-9b80-4b47a5243b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:08.727216 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d672e-7e5c-4d41-9b80-4b47a5243b12 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:08.727542 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d672e-7e5c-4d41-9b80-4b47a5243b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:08.727823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d672e-7e5c-4d41-9b80-4b47a5243b12 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:08.735544 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe4968-7e1b-4af9-a1ca-2f78bad01777 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:08.740032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d672e-7e5c-4d41-9b80-4b47a5243b12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:08.747857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-256d672e-7e5c-4d41-9b80-4b47a5243b12 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:13:08.747857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:08.747857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:08.747857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:08.747857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:08.747857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:08.747857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:08.747857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:08.747857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:08.747857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:08.747857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:08.747857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:08.747857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:08.747857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:08.747857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:08.747857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:08.748918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:08.748918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:08.748918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:08.748918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:08.748918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:08.748918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:08.748918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:08.748918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:08.748918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:08.748918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:08.748918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:08.748918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:08.748918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:08.748918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:08.748918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:08.748918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:08.749973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:08.749973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:08.749973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:08.749973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:08.749973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:08.749973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:08.749973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:08.749973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:08.749973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:08.749973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:08.749973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:13:08.749973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:08.749973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 151/303] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:08.749973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d672e-7e5c-4d41-9b80-4b47a5243b12 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:08.783527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc501821-5742-435b-8987-b1fb496cc8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:08.784700 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc501821-5742-435b-8987-b1fb496cc8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:08.785154 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc501821-5742-435b-8987-b1fb496cc8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:08.785571 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc501821-5742-435b-8987-b1fb496cc8f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:08.787586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc501821-5742-435b-8987-b1fb496cc8f5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:08.787938 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc501821-5742-435b-8987-b1fb496cc8f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:08.789818 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc501821-5742-435b-8987-b1fb496cc8f5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:08.802993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 153/304] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:13:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:08.803911 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc501821-5742-435b-8987-b1fb496cc8f5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:08.807458 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:08.863824 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d672e-7e5c-4d41-9b80-4b47a5243b12 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:08.897310 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:08.901132 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760eb29-673d-4e45-bb87-f6d491be9b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:08.901800 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760eb29-673d-4e45-bb87-f6d491be9b99 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:08.902083 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760eb29-673d-4e45-bb87-f6d491be9b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:08.902356 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760eb29-673d-4e45-bb87-f6d491be9b99 None admin] PATH_INFO: `/v3/groups` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:08.903044 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760eb29-673d-4e45-bb87-f6d491be9b99 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:08.904615 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760eb29-673d-4e45-bb87-f6d491be9b99 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:08.915098 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc501821-5742-435b-8987-b1fb496cc8f5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:08.952205 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646bd78b-11dd-4b9d-85d1-f778401cbfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:08.952885 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646bd78b-11dd-4b9d-85d1-f778401cbfdc None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:08.953172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646bd78b-11dd-4b9d-85d1-f778401cbfdc None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:08.953451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646bd78b-11dd-4b9d-85d1-f778401cbfdc None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:08.955066 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646bd78b-11dd-4b9d-85d1-f778401cbfdc None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:08.955375 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646bd78b-11dd-4b9d-85d1-f778401cbfdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:08.959262 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646bd78b-11dd-4b9d-85d1-f778401cbfdc None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:08.970721 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 154/305] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:13:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:08.972516 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646bd78b-11dd-4b9d-85d1-f778401cbfdc None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:08.991507 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 152/306] 10.222.0.67 () {60 vars in 1134 bytes} [Thu Jun 30 16:13:08 2022] POST /identity/v3/groups => generated 229 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:13:08.991507 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760eb29-673d-4e45-bb87-f6d491be9b99 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:09.134188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646bd78b-11dd-4b9d-85d1-f778401cbfdc None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:09.136657 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760eb29-673d-4e45-bb87-f6d491be9b99 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:09.146839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:09.184510 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fe9be-c2ee-4b53-8ea6-dbcbaaffbced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:09.185443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fe9be-c2ee-4b53-8ea6-dbcbaaffbced None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:09.185733 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fe9be-c2ee-4b53-8ea6-dbcbaaffbced None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:09.186090 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fe9be-c2ee-4b53-8ea6-dbcbaaffbced None admin] PATH_INFO: `/v3/users/f5d383dd1ece47deae3c9f504bc5afda` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:09.186944 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b45ab-31f9-4c30-8b4d-d22f77bbfe85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:09.187960 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b45ab-31f9-4c30-8b4d-d22f77bbfe85 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:09.188227 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b45ab-31f9-4c30-8b4d-d22f77bbfe85 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:09.188499 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b45ab-31f9-4c30-8b4d-d22f77bbfe85 None admin] PATH_INFO: `/v3/users/baee4e1d8ef14cc7871ee186f960e301` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:09.190542 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4b45ab-31f9-4c30-8b4d-d22f77bbfe85 None admin] RBAC: Authorizing `identity:get_user(user_id=baee4e1d8ef14cc7871ee186f960e301)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:09.191246 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4b45ab-31f9-4c30-8b4d-d22f77bbfe85 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:09.194352 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 153/307] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:13:08 2022] GET /identity/v3/users/baee4e1d8ef14cc7871ee186f960e301 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:09.200562 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b45ab-31f9-4c30-8b4d-d22f77bbfe85 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:09.206031 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814fe9be-c2ee-4b53-8ea6-dbcbaaffbced None admin] RBAC: Authorizing `identity:get_user(user_id=f5d383dd1ece47deae3c9f504bc5afda)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:09.207624 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814fe9be-c2ee-4b53-8ea6-dbcbaaffbced None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:09.209539 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 155/308] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:13:08 2022] GET /identity/v3/users/f5d383dd1ece47deae3c9f504bc5afda => generated 293 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:09.215542 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fe9be-c2ee-4b53-8ea6-dbcbaaffbced None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:09.220485 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:09.327502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b45ab-31f9-4c30-8b4d-d22f77bbfe85 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:09.337618 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fe9be-c2ee-4b53-8ea6-dbcbaaffbced None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:09.365438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708410c7-a067-4768-b2d8-d2b6df5a651d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:09.366601 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708410c7-a067-4768-b2d8-d2b6df5a651d None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:09.367909 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708410c7-a067-4768-b2d8-d2b6df5a651d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:09.368469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708410c7-a067-4768-b2d8-d2b6df5a651d None admin] PATH_INFO: `/v3/system/users/baee4e1d8ef14cc7871ee186f960e301/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:09.370178 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708410c7-a067-4768-b2d8-d2b6df5a651d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=baee4e1d8ef14cc7871ee186f960e301, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:09.371791 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708410c7-a067-4768-b2d8-d2b6df5a651d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:09.374872 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685e8855-420f-4b21-98ae-85e28ecf5b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:09.375587 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e8855-420f-4b21-98ae-85e28ecf5b67 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:09.375878 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e8855-420f-4b21-98ae-85e28ecf5b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:09.376460 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685e8855-420f-4b21-98ae-85e28ecf5b67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:09.377133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e8855-420f-4b21-98ae-85e28ecf5b67 None admin] RBAC: Adding query filter params (role.id=6e5a95d194ca42cab7a30baff4554fd3, user.id=f5d383dd1ece47deae3c9f504bc5afda, scope.system=all) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:09.377446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e8855-420f-4b21-98ae-85e28ecf5b67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:09.379085 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685e8855-420f-4b21-98ae-85e28ecf5b67 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:09.401800 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 156/309] 10.222.0.67 () {56 vars in 1308 bytes} [Thu Jun 30 16:13:09 2022] GET /identity/v3/role_assignments?user.id=f5d383dd1ece47deae3c9f504bc5afda&scope.system=all&role.id=6e5a95d194ca42cab7a30baff4554fd3 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:09.502118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 154/310] 10.222.0.67 () {58 vars in 1334 bytes} [Thu Jun 30 16:13:09 2022] PUT /identity/v3/system/users/baee4e1d8ef14cc7871ee186f960e301/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:13:09.626038 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c9f78-bfd3-4f49-b950-91a0055326ca None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:09.626363 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c9f78-bfd3-4f49-b950-91a0055326ca None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:09.626608 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c9f78-bfd3-4f49-b950-91a0055326ca None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:09.628346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 157/311] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:09.634320 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f5c21-98d8-4164-9cb8-adbf8fde8fd3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:09.634665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f5c21-98d8-4164-9cb8-adbf8fde8fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:09.634888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f5c21-98d8-4164-9cb8-adbf8fde8fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:09.654286 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-977f5c21-98d8-4164-9cb8-adbf8fde8fd3 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:09.655437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-977f5c21-98d8-4164-9cb8-adbf8fde8fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:09.692745 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:09.727553 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 155/312] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:09.752451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c9f78-bfd3-4f49-b950-91a0055326ca None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:09.829036 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:09.867834 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c9f78-bfd3-4f49-b950-91a0055326ca None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:09.904486 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df37271-9e03-4cba-ba09-2e4cb9f2c8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:09.905250 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df37271-9e03-4cba-ba09-2e4cb9f2c8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:09.905537 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df37271-9e03-4cba-ba09-2e4cb9f2c8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:09.905813 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df37271-9e03-4cba-ba09-2e4cb9f2c8e3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:09.917405 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df37271-9e03-4cba-ba09-2e4cb9f2c8e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:09.925568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-6df37271-9e03-4cba-ba09-2e4cb9f2c8e3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 16:13:09.925568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:09.925568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:09.925568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:09.925568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:09.925568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:09.925568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:09.925568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:09.925568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:09.925568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:09.925568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:09.925568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:09.925568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:09.925568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:09.925568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:09.925568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:09.926467 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:09.926467 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:09.926467 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:09.926467 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:09.926467 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:09.926467 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:09.926467 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:09.926467 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:09.926467 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:09.926467 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:09.926467 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:09.926467 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:09.926467 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:09.926467 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:09.926467 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:09.926467 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:09.927290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:09.927290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:09.927290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:09.927290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:09.927290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:09.927290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:09.927290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:09.927290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:09.927290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:09.927290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:09.927290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 16:13:09.927290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:09.927290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 158/313] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:09.933494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f5c21-98d8-4164-9cb8-adbf8fde8fd3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:10.040021 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f5c21-98d8-4164-9cb8-adbf8fde8fd3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:10.076205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68efca4-4666-4d36-85b7-e44159bef046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:10.077280 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68efca4-4666-4d36-85b7-e44159bef046 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:10.077766 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68efca4-4666-4d36-85b7-e44159bef046 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:10.078258 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68efca4-4666-4d36-85b7-e44159bef046 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:10.079217 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68efca4-4666-4d36-85b7-e44159bef046 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:10.079745 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68efca4-4666-4d36-85b7-e44159bef046 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:10.081376 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68efca4-4666-4d36-85b7-e44159bef046 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:10.090528 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 156/314] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:13:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:10.096784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df37271-9e03-4cba-ba09-2e4cb9f2c8e3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:10.211603 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df37271-9e03-4cba-ba09-2e4cb9f2c8e3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:10.240186 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:10.248768 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7155dd0-2187-4c94-a284-76515da25af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:10.252570 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7155dd0-2187-4c94-a284-76515da25af3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:10.252570 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7155dd0-2187-4c94-a284-76515da25af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:10.252570 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7155dd0-2187-4c94-a284-76515da25af3 None admin] PATH_INFO: `/v3/users/eacedf5ae1aa44e1aa6dac0ebedf4b57` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:10.252570 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7155dd0-2187-4c94-a284-76515da25af3 None admin] RBAC: Authorizing `identity:get_user(user_id=eacedf5ae1aa44e1aa6dac0ebedf4b57)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:10.253191 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7155dd0-2187-4c94-a284-76515da25af3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:10.255140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 159/315] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:13:10 2022] GET /identity/v3/users/eacedf5ae1aa44e1aa6dac0ebedf4b57 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:10.264570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68efca4-4666-4d36-85b7-e44159bef046 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:10.302490 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:10.375011 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68efca4-4666-4d36-85b7-e44159bef046 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:10.410300 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb49b908-5b8a-48b4-acb9-b4b73f60e537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:10.411047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49b908-5b8a-48b4-acb9-b4b73f60e537 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:10.411262 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49b908-5b8a-48b4-acb9-b4b73f60e537 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:10.411571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb49b908-5b8a-48b4-acb9-b4b73f60e537 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:10.482769 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb49b908-5b8a-48b4-acb9-b4b73f60e537 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:10.484304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb49b908-5b8a-48b4-acb9-b4b73f60e537 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:10.515238 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:10.591667 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 157/316] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:13:10.597110 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7155dd0-2187-4c94-a284-76515da25af3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:10.708885 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7155dd0-2187-4c94-a284-76515da25af3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:10.747442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69cb9d-e133-4b98-beda-3cb672101fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:10.748257 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69cb9d-e133-4b98-beda-3cb672101fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:10.748538 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69cb9d-e133-4b98-beda-3cb672101fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:10.748814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69cb9d-e133-4b98-beda-3cb672101fd8 None admin] PATH_INFO: `/v3/projects/195eb382a49146b6b27a5cdc500c1a05` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:10.750008 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff69cb9d-e133-4b98-beda-3cb672101fd8 None admin] RBAC: Authorizing `identity:get_project(project_id=195eb382a49146b6b27a5cdc500c1a05)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:10.751491 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff69cb9d-e133-4b98-beda-3cb672101fd8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:10.753515 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 160/317] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:13:10 2022] GET /identity/v3/projects/195eb382a49146b6b27a5cdc500c1a05 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:10.759795 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb49b908-5b8a-48b4-acb9-b4b73f60e537 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:10.865641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb49b908-5b8a-48b4-acb9-b4b73f60e537 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:10.896352 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:10.902454 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32267497-f4fd-47a8-9165-8d98e7abc1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:10.903143 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32267497-f4fd-47a8-9165-8d98e7abc1dd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:10.903725 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32267497-f4fd-47a8-9165-8d98e7abc1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:10.904035 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32267497-f4fd-47a8-9165-8d98e7abc1dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:10.904882 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32267497-f4fd-47a8-9165-8d98e7abc1dd None admin] RBAC: Adding query filter params (role.id=6e5a95d194ca42cab7a30baff4554fd3, scope.project.id=195eb382a49146b6b27a5cdc500c1a05, user.id=eacedf5ae1aa44e1aa6dac0ebedf4b57) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:10.905200 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32267497-f4fd-47a8-9165-8d98e7abc1dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:10.906676 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32267497-f4fd-47a8-9165-8d98e7abc1dd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:10.913404 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 158/318] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:13:10 2022] GET /identity/v3/role_assignments?user.id=eacedf5ae1aa44e1aa6dac0ebedf4b57&scope.project.id=195eb382a49146b6b27a5cdc500c1a05&role.id=6e5a95d194ca42cab7a30baff4554fd3 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:11.170333 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736683c-ae27-483d-b701-4569e1e5a063 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:11.170630 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736683c-ae27-483d-b701-4569e1e5a063 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:11.170918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736683c-ae27-483d-b701-4569e1e5a063 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:11.172335 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 161/319] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:11.177583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf3e96-669c-494b-8b21-0364252d2376 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:11.177877 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf3e96-669c-494b-8b21-0364252d2376 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:11.178155 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf3e96-669c-494b-8b21-0364252d2376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:11.195100 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4daf3e96-669c-494b-8b21-0364252d2376 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:11.196345 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4daf3e96-669c-494b-8b21-0364252d2376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:11.270234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 159/320] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:11.295303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a736683c-ae27-483d-b701-4569e1e5a063 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:11.360418 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:11.399702 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a736683c-ae27-483d-b701-4569e1e5a063 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:11.440050 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154eae54-1772-4039-8ffd-889301f58a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:11.440050 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154eae54-1772-4039-8ffd-889301f58a8a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:11.440050 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154eae54-1772-4039-8ffd-889301f58a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:11.440050 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154eae54-1772-4039-8ffd-889301f58a8a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:11.440526 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154eae54-1772-4039-8ffd-889301f58a8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:11.440526 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154eae54-1772-4039-8ffd-889301f58a8a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:11.441753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 162/321] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:11.448471 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf3e96-669c-494b-8b21-0364252d2376 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:11.564980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ef960-41a4-42a0-bf93-80e1f8690a68 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:11.565275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ef960-41a4-42a0-bf93-80e1f8690a68 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:11.565554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ef960-41a4-42a0-bf93-80e1f8690a68 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:11.566927 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 163/322] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:13:11.572643 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5415836-e57f-4305-972a-efbd6ddd2cad None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:11.572927 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5415836-e57f-4305-972a-efbd6ddd2cad None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:11.573214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5415836-e57f-4305-972a-efbd6ddd2cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:11.575728 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4daf3e96-669c-494b-8b21-0364252d2376 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:11.592143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c5415836-e57f-4305-972a-efbd6ddd2cad None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:11.593324 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c5415836-e57f-4305-972a-efbd6ddd2cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:11.604824 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:11.614681 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac8fb3-fcf4-49c5-a48b-d053bcfcbe62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:11.615409 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac8fb3-fcf4-49c5-a48b-d053bcfcbe62 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:11.615658 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac8fb3-fcf4-49c5-a48b-d053bcfcbe62 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:11.615928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac8fb3-fcf4-49c5-a48b-d053bcfcbe62 None admin] PATH_INFO: `/v3/groups` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:11.616633 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac8fb3-fcf4-49c5-a48b-d053bcfcbe62 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:11.618922 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac8fb3-fcf4-49c5-a48b-d053bcfcbe62 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:11.662111 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 164/323] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:13:11.684546 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5415836-e57f-4305-972a-efbd6ddd2cad None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:11.745392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:11.751058 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 160/324] 10.222.0.67 () {60 vars in 1134 bytes} [Thu Jun 30 16:13:11 2022] POST /identity/v3/groups => generated 226 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:13:11.751058 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198d6d0-ca6e-4e2f-989c-f5db91e471cb None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:11.751058 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198d6d0-ca6e-4e2f-989c-f5db91e471cb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:11.751058 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1198d6d0-ca6e-4e2f-989c-f5db91e471cb None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:11.751058 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 161/325] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:11.765375 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870d4ec-83fe-44f4-aa8d-e95174f8ba14 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:11.765700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870d4ec-83fe-44f4-aa8d-e95174f8ba14 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:11.765979 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870d4ec-83fe-44f4-aa8d-e95174f8ba14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:11.785583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8870d4ec-83fe-44f4-aa8d-e95174f8ba14 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:11.789792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8870d4ec-83fe-44f4-aa8d-e95174f8ba14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:11.797262 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5415836-e57f-4305-972a-efbd6ddd2cad None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:11.844765 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f832f-3b90-4b81-bd38-a856354c0873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:11.846324 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f832f-3b90-4b81-bd38-a856354c0873 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:11.846613 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f832f-3b90-4b81-bd38-a856354c0873 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:11.846887 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f832f-3b90-4b81-bd38-a856354c0873 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:11.860836 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f832f-3b90-4b81-bd38-a856354c0873 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:11.877511 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 162/326] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:11.877703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-522f832f-3b90-4b81-bd38-a856354c0873 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 16:13:11.877703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:11.877703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:11.877703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:11.877703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:11.877703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:11.877703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:11.877703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:11.877703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:11.877703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:11.877703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:11.877703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:11.877703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:11.877703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:11.877703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:11.877703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:11.878520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:11.878520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:11.878520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:11.878520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:11.878520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:11.878520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:11.878520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:11.878520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:11.878520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:11.878520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:11.878520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:11.878520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:11.878520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:11.878520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:11.878520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:11.878520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:11.879271 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:11.879271 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:11.879271 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:11.879271 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:11.879271 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:11.879271 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:11.879271 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:11.879271 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:11.879271 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:11.879271 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:11.879271 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 16:13:11.879271 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:11.879271 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 165/327] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:13:11.879952 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8870d4ec-83fe-44f4-aa8d-e95174f8ba14 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:11.901666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f832f-3b90-4b81-bd38-a856354c0873 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:11.994934 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8870d4ec-83fe-44f4-aa8d-e95174f8ba14 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:12.020680 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:12.040141 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f832f-3b90-4b81-bd38-a856354c0873 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:12.042375 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5742314-cdc9-40f4-b448-6238d0fa7348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:12.043052 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5742314-cdc9-40f4-b448-6238d0fa7348 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:12.043335 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5742314-cdc9-40f4-b448-6238d0fa7348 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:12.043642 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5742314-cdc9-40f4-b448-6238d0fa7348 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:12.044390 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5742314-cdc9-40f4-b448-6238d0fa7348 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:12.044698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5742314-cdc9-40f4-b448-6238d0fa7348 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:12.046093 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5742314-cdc9-40f4-b448-6238d0fa7348 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:12.057425 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 163/328] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:13:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:12.063102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5742314-cdc9-40f4-b448-6238d0fa7348 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:12.071202 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:12.089956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b6e991-f81a-48b7-b8d0-6e56aa105f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:12.091099 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6e991-f81a-48b7-b8d0-6e56aa105f4d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:12.091441 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6e991-f81a-48b7-b8d0-6e56aa105f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:12.091830 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6e991-f81a-48b7-b8d0-6e56aa105f4d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:12.104934 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b6e991-f81a-48b7-b8d0-6e56aa105f4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:12.112866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-b3b6e991-f81a-48b7-b8d0-6e56aa105f4d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:13:12.112866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:12.112866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:12.112866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:12.112866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:12.112866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:12.112866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:12.112866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:12.112866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:12.112866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:12.112866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:12.112866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:12.112866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:12.112866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:12.112866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:12.112866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:12.113852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:12.113852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:12.113852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:12.113852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:12.113852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:12.113852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:12.113852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:12.113852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:12.113852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:12.113852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:12.113852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:12.113852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:12.113852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:12.113852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:12.113852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:12.113852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:12.114707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:12.114707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:12.114707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:12.114707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:12.114707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:12.114707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:12.114707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:12.114707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:12.114707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:12.114707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:12.114707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:13:12.114707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:12.114707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 166/329] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:13:12.125446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b6e991-f81a-48b7-b8d0-6e56aa105f4d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:12.212191 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5742314-cdc9-40f4-b448-6238d0fa7348 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:12.254963 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b6e991-f81a-48b7-b8d0-6e56aa105f4d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:12.265443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024de906-a6aa-49ca-84d3-b52f0e2579e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:12.266897 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024de906-a6aa-49ca-84d3-b52f0e2579e6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:12.267196 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024de906-a6aa-49ca-84d3-b52f0e2579e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:12.267503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024de906-a6aa-49ca-84d3-b52f0e2579e6 None admin] PATH_INFO: `/v3/users/f5d383dd1ece47deae3c9f504bc5afda` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:12.269232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024de906-a6aa-49ca-84d3-b52f0e2579e6 None admin] RBAC: Authorizing `identity:get_user(user_id=f5d383dd1ece47deae3c9f504bc5afda)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:12.271006 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024de906-a6aa-49ca-84d3-b52f0e2579e6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:12.274517 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 164/330] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:13:12 2022] GET /identity/v3/users/f5d383dd1ece47deae3c9f504bc5afda => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:12.279725 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024de906-a6aa-49ca-84d3-b52f0e2579e6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:12.307826 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc51a5-70c3-4dcc-9ac0-e4b07e947854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:12.308809 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc51a5-70c3-4dcc-9ac0-e4b07e947854 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:12.309586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc51a5-70c3-4dcc-9ac0-e4b07e947854 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:12.310026 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc51a5-70c3-4dcc-9ac0-e4b07e947854 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:12.311226 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc51a5-70c3-4dcc-9ac0-e4b07e947854 None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:12.312030 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc51a5-70c3-4dcc-9ac0-e4b07e947854 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:12.314162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc51a5-70c3-4dcc-9ac0-e4b07e947854 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:12.336202 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 167/331] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:13:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:12.340274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc51a5-70c3-4dcc-9ac0-e4b07e947854 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:12.395273 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:12.405110 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024de906-a6aa-49ca-84d3-b52f0e2579e6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:12.448935 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92df3437-8a89-40a7-9c25-ed6dbd4f4583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:12.451312 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df3437-8a89-40a7-9c25-ed6dbd4f4583 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:12.451846 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df3437-8a89-40a7-9c25-ed6dbd4f4583 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:12.452756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92df3437-8a89-40a7-9c25-ed6dbd4f4583 None admin] PATH_INFO: `/v3/system/users/f5d383dd1ece47deae3c9f504bc5afda/roles/6e5a95d194ca42cab7a30baff4554fd3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:12.455168 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df3437-8a89-40a7-9c25-ed6dbd4f4583 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f5d383dd1ece47deae3c9f504bc5afda, role_id=6e5a95d194ca42cab7a30baff4554fd3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:12.457654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92df3437-8a89-40a7-9c25-ed6dbd4f4583 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:12.464567 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc51a5-70c3-4dcc-9ac0-e4b07e947854 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:12.495254 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:12.505605 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b79efa-b922-4501-834b-c0bae16e6b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:12.506033 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b79efa-b922-4501-834b-c0bae16e6b34 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:12.506312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b79efa-b922-4501-834b-c0bae16e6b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:12.506584 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b79efa-b922-4501-834b-c0bae16e6b34 None admin] PATH_INFO: `/v3/users/baee4e1d8ef14cc7871ee186f960e301` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:12.508761 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b79efa-b922-4501-834b-c0bae16e6b34 None admin] RBAC: Authorizing `identity:get_user(user_id=baee4e1d8ef14cc7871ee186f960e301)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:12.510213 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b79efa-b922-4501-834b-c0bae16e6b34 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:12.512158 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 168/332] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:13:12 2022] GET /identity/v3/users/baee4e1d8ef14cc7871ee186f960e301 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:12.518241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b79efa-b922-4501-834b-c0bae16e6b34 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:12.631927 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b79efa-b922-4501-834b-c0bae16e6b34 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:12.645729 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:12.669770 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a3b5d5-af80-4614-b665-12c380869a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:12.670571 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3b5d5-af80-4614-b665-12c380869a99 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:12.670759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3b5d5-af80-4614-b665-12c380869a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:12.671037 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a3b5d5-af80-4614-b665-12c380869a99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:12.672019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3b5d5-af80-4614-b665-12c380869a99 None admin] RBAC: Adding query filter params (role.id=9ab3e302ead54950b7c30103f1e7fda7, user.id=baee4e1d8ef14cc7871ee186f960e301, scope.system=all) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:12.672345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3b5d5-af80-4614-b665-12c380869a99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:12.673806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a3b5d5-af80-4614-b665-12c380869a99 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:12.677884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 165/333] 10.222.0.67 () {58 vars in 1334 bytes} [Thu Jun 30 16:13:12 2022] PUT /identity/v3/system/users/f5d383dd1ece47deae3c9f504bc5afda/roles/6e5a95d194ca42cab7a30baff4554fd3 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:13:12.691063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 169/334] 10.222.0.67 () {56 vars in 1308 bytes} [Thu Jun 30 16:13:12 2022] GET /identity/v3/role_assignments?user.id=baee4e1d8ef14cc7871ee186f960e301&scope.system=all&role.id=9ab3e302ead54950b7c30103f1e7fda7 => generated 510 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:14.630589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43c412-7452-422a-ba89-13c3a231d3f4 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:14.630589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43c412-7452-422a-ba89-13c3a231d3f4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:14.630589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d43c412-7452-422a-ba89-13c3a231d3f4 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:14.630589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 166/335] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:14.640915 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556a0d4-e3df-4f0b-81a9-ddd72055643b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:14.641905 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556a0d4-e3df-4f0b-81a9-ddd72055643b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:14.642407 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556a0d4-e3df-4f0b-81a9-ddd72055643b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:14.642619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede844c4-8a1c-49e4-b22a-e7fb7bec9823 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:14.642905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede844c4-8a1c-49e4-b22a-e7fb7bec9823 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:14.643214 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede844c4-8a1c-49e4-b22a-e7fb7bec9823 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:14.644833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 167/336] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:14.653738 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb584c8-319d-43e9-92c0-3f6695394d77 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:14.654017 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb584c8-319d-43e9-92c0-3f6695394d77 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:14.654504 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb584c8-319d-43e9-92c0-3f6695394d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:14.663636 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c556a0d4-e3df-4f0b-81a9-ddd72055643b None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:14.665210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c556a0d4-e3df-4f0b-81a9-ddd72055643b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:14.684280 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-edb584c8-319d-43e9-92c0-3f6695394d77 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:14.685480 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-edb584c8-319d-43e9-92c0-3f6695394d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:14.755737 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 170/337] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:13:14.781336 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556a0d4-e3df-4f0b-81a9-ddd72055643b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:14.857427 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 168/338] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:13:14.895394 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556a0d4-e3df-4f0b-81a9-ddd72055643b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:14.905584 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb584c8-319d-43e9-92c0-3f6695394d77 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:14.911708 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:14.936898 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e4fb1-2f0b-4727-8a40-9078fd703262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:14.938064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e4fb1-2f0b-4727-8a40-9078fd703262 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:14.938568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e4fb1-2f0b-4727-8a40-9078fd703262 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:14.939257 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e4fb1-2f0b-4727-8a40-9078fd703262 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:14.951571 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8e4fb1-2f0b-4727-8a40-9078fd703262 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:14.961104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ed8e4fb1-2f0b-4727-8a40-9078fd703262 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:13:14.961104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:14.961104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:14.961104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:14.961104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:14.961104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:14.961104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:14.961104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:14.961104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:14.961104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:14.961104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:14.961104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:14.961104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:14.961104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:14.961104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:14.961104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:14.962028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:14.962028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:14.962028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:14.962028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:14.962028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:14.962028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:14.962028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:14.962028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:14.962028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:14.962028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:14.962028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:14.962028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:14.962028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:14.962028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:14.962028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:14.962028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:14.962801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:14.962801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:14.962801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:14.962801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:14.962801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:14.962801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:14.962801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:14.962801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:14.962801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:14.962801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:14.962801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:13:14.962801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:14.970752 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 171/339] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:14.972467 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e4fb1-2f0b-4727-8a40-9078fd703262 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:15.050143 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:15.087761 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e4fb1-2f0b-4727-8a40-9078fd703262 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:15.125850 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d877fca-5086-48a8-bdc2-637f7cca96c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:15.126983 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d877fca-5086-48a8-bdc2-637f7cca96c0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:15.127500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d877fca-5086-48a8-bdc2-637f7cca96c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:15.127997 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d877fca-5086-48a8-bdc2-637f7cca96c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:15.129298 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d877fca-5086-48a8-bdc2-637f7cca96c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:15.129831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d877fca-5086-48a8-bdc2-637f7cca96c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:15.131564 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d877fca-5086-48a8-bdc2-637f7cca96c0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:15.147187 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 172/340] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:13:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:15.149036 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d877fca-5086-48a8-bdc2-637f7cca96c0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:15.177535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:15.250673 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb584c8-319d-43e9-92c0-3f6695394d77 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:15.267425 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d877fca-5086-48a8-bdc2-637f7cca96c0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:15.309146 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88395223-8b54-4628-a2fa-9d3e85a2a519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:15.310172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88395223-8b54-4628-a2fa-9d3e85a2a519 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:15.310625 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88395223-8b54-4628-a2fa-9d3e85a2a519 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:15.311053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88395223-8b54-4628-a2fa-9d3e85a2a519 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:15.380236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d976f-f17c-4684-9ed4-dcdff1de4a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:15.381048 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d976f-f17c-4684-9ed4-dcdff1de4a64 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:15.381563 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d976f-f17c-4684-9ed4-dcdff1de4a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:15.382082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d976f-f17c-4684-9ed4-dcdff1de4a64 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:15.388586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88395223-8b54-4628-a2fa-9d3e85a2a519 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:15.390374 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88395223-8b54-4628-a2fa-9d3e85a2a519 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:15.405041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d976f-f17c-4684-9ed4-dcdff1de4a64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:15.421118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-105d976f-f17c-4684-9ed4-dcdff1de4a64 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:13:15.421118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:15.421118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:15.421118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:15.421118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:15.421118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:15.421118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:15.421118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:15.421118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:15.421118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:15.421118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:15.421118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:15.421118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:15.421118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:15.421118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:15.421118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:15.422023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:15.422023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:15.422023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:15.422023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:15.422023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:15.422023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:15.422023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:15.422023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:15.422023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:15.422023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:15.422023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:15.422023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:15.422023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:15.422023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:15.422023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:15.422023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:15.422777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:15.422777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:15.422777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:15.422777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:15.422777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:15.422777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:15.422777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:15.422777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:15.422777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:15.422777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:15.422777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:13:15.422777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:15.425445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 169/341] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 520 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:15.432165 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d976f-f17c-4684-9ed4-dcdff1de4a64 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:15.434009 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:15.510316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 173/342] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:13:15.511855 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88395223-8b54-4628-a2fa-9d3e85a2a519 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:15.595955 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:15.603162 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d976f-f17c-4684-9ed4-dcdff1de4a64 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:15.626261 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88395223-8b54-4628-a2fa-9d3e85a2a519 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:15.663425 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:15.668290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742a1e6-75c8-4b78-9bbc-f98fd6be288c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:15.669171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742a1e6-75c8-4b78-9bbc-f98fd6be288c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:15.669453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742a1e6-75c8-4b78-9bbc-f98fd6be288c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:15.669824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742a1e6-75c8-4b78-9bbc-f98fd6be288c None admin] PATH_INFO: `/v3/projects/e54d8744be3c4672b24fa2e19f024af0` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:15.671041 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742a1e6-75c8-4b78-9bbc-f98fd6be288c None admin] RBAC: Authorizing `identity:get_project(project_id=e54d8744be3c4672b24fa2e19f024af0)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:15.672539 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742a1e6-75c8-4b78-9bbc-f98fd6be288c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:15.674993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 174/343] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:13:15 2022] GET /identity/v3/projects/e54d8744be3c4672b24fa2e19f024af0 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:15.677073 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06771e55-bc2c-420f-8020-11d7231423f5 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:15.677513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06771e55-bc2c-420f-8020-11d7231423f5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:15.677957 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06771e55-bc2c-420f-8020-11d7231423f5 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:15.680614 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 175/344] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:15.681531 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06771e55-bc2c-420f-8020-11d7231423f5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:15.731954 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565aba08-9b2c-425b-990f-760e0c14f502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:15.733350 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565aba08-9b2c-425b-990f-760e0c14f502 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:15.738532 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565aba08-9b2c-425b-990f-760e0c14f502 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:15.739046 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565aba08-9b2c-425b-990f-760e0c14f502 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:15.740044 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565aba08-9b2c-425b-990f-760e0c14f502 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:15.740513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565aba08-9b2c-425b-990f-760e0c14f502 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:15.742633 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565aba08-9b2c-425b-990f-760e0c14f502 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:15.763452 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 170/345] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:13:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:15.765262 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587fd95-d070-4ad1-8393-a49d9a6250dc None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:15.765771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587fd95-d070-4ad1-8393-a49d9a6250dc None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:15.766262 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587fd95-d070-4ad1-8393-a49d9a6250dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:15.797681 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5587fd95-d070-4ad1-8393-a49d9a6250dc None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:15.807502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5587fd95-d070-4ad1-8393-a49d9a6250dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:15.831741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06771e55-bc2c-420f-8020-11d7231423f5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:15.914907 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c503f1-1df0-4cd2-b8e7-4dac828abcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:15.915847 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c503f1-1df0-4cd2-b8e7-4dac828abcd1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:15.916135 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c503f1-1df0-4cd2-b8e7-4dac828abcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:15.916336 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 171/346] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:15.916434 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c503f1-1df0-4cd2-b8e7-4dac828abcd1 None admin] PATH_INFO: `/v3/groups/026420b86aa24c14807e7f9fce4ef606` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:15.917178 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587fd95-d070-4ad1-8393-a49d9a6250dc None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:15.924121 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c503f1-1df0-4cd2-b8e7-4dac828abcd1 None admin] RBAC: Authorizing `identity:get_group(group_id=026420b86aa24c14807e7f9fce4ef606)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:15.925587 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c503f1-1df0-4cd2-b8e7-4dac828abcd1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:15.927643 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 176/347] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 30 16:13:15 2022] GET /identity/v3/groups/026420b86aa24c14807e7f9fce4ef606 => generated 226 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:15.928909 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6463f5-8607-41a8-979c-4f240f39c463 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:15.930348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6463f5-8607-41a8-979c-4f240f39c463 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:15.930634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6463f5-8607-41a8-979c-4f240f39c463 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:15.932156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 177/348] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:15.936638 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6463f5-8607-41a8-979c-4f240f39c463 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:16.007917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:16.023107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:16.033389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587fd95-d070-4ad1-8393-a49d9a6250dc None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:16.072135 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a48805-46d8-4b7b-be06-3d2a2d731664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:16.072822 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a48805-46d8-4b7b-be06-3d2a2d731664 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:16.073280 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a48805-46d8-4b7b-be06-3d2a2d731664 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:16.073700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a48805-46d8-4b7b-be06-3d2a2d731664 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:16.074793 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6463f5-8607-41a8-979c-4f240f39c463 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:16.117144 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29acd7-c109-45f1-9d80-0647334edaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:16.117807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29acd7-c109-45f1-9d80-0647334edaaf None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:16.118099 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29acd7-c109-45f1-9d80-0647334edaaf None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:16.118547 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29acd7-c109-45f1-9d80-0647334edaaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:16.120264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29acd7-c109-45f1-9d80-0647334edaaf None admin] RBAC: Adding query filter params (group.id=026420b86aa24c14807e7f9fce4ef606, role.id=9ab3e302ead54950b7c30103f1e7fda7, scope.project.id=e54d8744be3c4672b24fa2e19f024af0) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:16.120587 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29acd7-c109-45f1-9d80-0647334edaaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:16.122083 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29acd7-c109-45f1-9d80-0647334edaaf None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:16.129200 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 178/349] 10.222.0.67 () {56 vars in 1376 bytes} [Thu Jun 30 16:13:15 2022] GET /identity/v3/role_assignments?group.id=026420b86aa24c14807e7f9fce4ef606&scope.project.id=e54d8744be3c4672b24fa2e19f024af0&role.id=9ab3e302ead54950b7c30103f1e7fda7 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:16.131383 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3377a-71ab-4b91-a8f2-9cad15721036 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:16.131886 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3377a-71ab-4b91-a8f2-9cad15721036 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:16.132858 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3377a-71ab-4b91-a8f2-9cad15721036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:16.154752 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-87a3377a-71ab-4b91-a8f2-9cad15721036 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:16.156274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-87a3377a-71ab-4b91-a8f2-9cad15721036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:16.163063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a48805-46d8-4b7b-be06-3d2a2d731664 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:16.164813 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a48805-46d8-4b7b-be06-3d2a2d731664 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:16.253025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 179/350] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:16.254259 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3377a-71ab-4b91-a8f2-9cad15721036 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:16.280483 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:16.293062 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 172/351] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:13:16.300273 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37d53b-4e3e-4cc5-9f52-2d3587ed4aa8 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:16.300273 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37d53b-4e3e-4cc5-9f52-2d3587ed4aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:16.300273 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37d53b-4e3e-4cc5-9f52-2d3587ed4aa8 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:16.300273 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 173/352] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:16.300273 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37d53b-4e3e-4cc5-9f52-2d3587ed4aa8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:16.333620 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:16.406541 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3377a-71ab-4b91-a8f2-9cad15721036 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:16.433203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37d53b-4e3e-4cc5-9f52-2d3587ed4aa8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:16.453235 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08508496-16f9-4235-9354-c4348ad6fa48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:16.454429 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08508496-16f9-4235-9354-c4348ad6fa48 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:16.454920 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08508496-16f9-4235-9354-c4348ad6fa48 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:16.455404 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08508496-16f9-4235-9354-c4348ad6fa48 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:16.470902 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08508496-16f9-4235-9354-c4348ad6fa48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:16.474731 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e9cb99-ea9d-4926-b48d-3ca5a1c3eeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:16.475905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9cb99-ea9d-4926-b48d-3ca5a1c3eeb5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:16.476441 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9cb99-ea9d-4926-b48d-3ca5a1c3eeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:16.476925 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9cb99-ea9d-4926-b48d-3ca5a1c3eeb5 None admin] PATH_INFO: `/v3/projects/195eb382a49146b6b27a5cdc500c1a05` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:16.478338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e9cb99-ea9d-4926-b48d-3ca5a1c3eeb5 None admin] RBAC: Authorizing `identity:get_project(project_id=195eb382a49146b6b27a5cdc500c1a05)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:16.480041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e9cb99-ea9d-4926-b48d-3ca5a1c3eeb5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:16.480162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-08508496-16f9-4235-9354-c4348ad6fa48 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:13:16.480162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:16.480162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:16.480162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:16.480162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:16.480162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:16.480162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:16.480162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:16.480162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:16.480162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:16.480162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:16.480162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:16.480162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:16.480162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:16.480162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:16.480162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:16.481005 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:16.481005 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:16.481005 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:16.481005 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:16.481005 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:16.481005 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:16.481005 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:16.481005 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:16.481005 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:16.481005 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:16.481005 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:16.481005 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:16.481005 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:16.481005 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:16.481005 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:16.481005 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:16.481807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:16.481807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:16.481807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:16.481807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:16.481807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:16.481807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:16.481807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:16.481807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:16.481807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:16.481807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:16.481807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:13:16.481807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:16.481807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 180/353] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 30 16:13:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:16.483002 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08508496-16f9-4235-9354-c4348ad6fa48 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:16.485533 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 174/354] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:13:16 2022] GET /identity/v3/projects/195eb382a49146b6b27a5cdc500c1a05 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:16.488044 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ace2fc-877d-416d-a7eb-edc5f5cd84be None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:16.488541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ace2fc-877d-416d-a7eb-edc5f5cd84be None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:16.489034 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ace2fc-877d-416d-a7eb-edc5f5cd84be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:16.511042 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-11ace2fc-877d-416d-a7eb-edc5f5cd84be None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:16.513536 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-11ace2fc-877d-416d-a7eb-edc5f5cd84be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:16.594025 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:16.609565 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 175/355] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:16.611334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ace2fc-877d-416d-a7eb-edc5f5cd84be None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:16.613203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08508496-16f9-4235-9354-c4348ad6fa48 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:16.625939 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:16.658163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406965ab-6aa7-402d-a512-19b716975e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:16.658604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406965ab-6aa7-402d-a512-19b716975e09 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:16.659018 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406965ab-6aa7-402d-a512-19b716975e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:16.659365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406965ab-6aa7-402d-a512-19b716975e09 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:16.672387 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406965ab-6aa7-402d-a512-19b716975e09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:16.684022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-406965ab-6aa7-402d-a512-19b716975e09 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 16:13:16.684022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:16.684022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:16.684022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:16.684022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:16.684022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:16.684022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:16.684022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:16.684022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:16.684022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:16.684022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:16.684022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:16.684022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:16.684022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:16.684022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:16.684022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:16.684958 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:16.684958 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:16.684958 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:16.684958 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:16.684958 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:16.684958 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:16.684958 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:16.684958 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:16.684958 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:16.684958 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:16.684958 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:16.684958 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:16.684958 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:16.684958 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:16.684958 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:16.684958 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:16.685727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:16.685727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:16.685727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:16.685727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:16.685727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:16.685727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:16.685727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:16.685727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:16.685727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:16.685727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:16.685727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 16:13:16.685727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:16.690597 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 181/356] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:16.691306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406965ab-6aa7-402d-a512-19b716975e09 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:16.754386 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ace2fc-877d-416d-a7eb-edc5f5cd84be None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:16.797797 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16f6db-fed2-4204-b282-7fe5fb62c653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:16.798940 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16f6db-fed2-4204-b282-7fe5fb62c653 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:16.800307 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16f6db-fed2-4204-b282-7fe5fb62c653 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:16.800808 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16f6db-fed2-4204-b282-7fe5fb62c653 None admin] PATH_INFO: `/v3/groups/026420b86aa24c14807e7f9fce4ef606` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:16.802201 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16f6db-fed2-4204-b282-7fe5fb62c653 None admin] RBAC: Authorizing `identity:get_group(group_id=026420b86aa24c14807e7f9fce4ef606)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:16.803898 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16f6db-fed2-4204-b282-7fe5fb62c653 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:16.806801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 176/357] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 30 16:13:16 2022] GET /identity/v3/groups/026420b86aa24c14807e7f9fce4ef606 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:16.807781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16f6db-fed2-4204-b282-7fe5fb62c653 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:16.817133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406965ab-6aa7-402d-a512-19b716975e09 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:16.861962 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3530a4-20bb-44ba-8688-40f720a7aa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:16.862638 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3530a4-20bb-44ba-8688-40f720a7aa2a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:16.862914 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3530a4-20bb-44ba-8688-40f720a7aa2a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:16.863194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3530a4-20bb-44ba-8688-40f720a7aa2a None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:16.864004 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3530a4-20bb-44ba-8688-40f720a7aa2a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:16.864432 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3530a4-20bb-44ba-8688-40f720a7aa2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:16.865845 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3530a4-20bb-44ba-8688-40f720a7aa2a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:16.876331 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 182/358] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:13:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:16.877189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3530a4-20bb-44ba-8688-40f720a7aa2a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:16.878693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:16.916012 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:16.925744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16f6db-fed2-4204-b282-7fe5fb62c653 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:16.967946 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7d16b-5bec-49d4-827a-de3b8e9ac9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:16.969394 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7d16b-5bec-49d4-827a-de3b8e9ac9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:16.969743 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7d16b-5bec-49d4-827a-de3b8e9ac9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:16.970087 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7d16b-5bec-49d4-827a-de3b8e9ac9b2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:16.984893 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7d16b-5bec-49d4-827a-de3b8e9ac9b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:16.995408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f5d7d16b-5bec-49d4-827a-de3b8e9ac9b2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:13:16.995408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:16.995408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:16.995408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:16.995408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:16.995408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:16.995408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:16.995408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:16.995408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:16.995408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:16.995408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:16.995408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:16.995408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:16.995408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:16.995408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:16.995408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:16.996367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:16.996367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:16.996367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:16.996367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:16.996367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:16.996367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:16.996367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:16.996367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:16.996367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:16.996367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:16.996367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:16.996367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:16.996367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:16.996367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:16.996367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:16.996367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:16.997186 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:16.997186 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:16.997186 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:16.997186 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:16.997186 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:16.997186 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:16.997186 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:16.997186 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:16.997186 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:16.997186 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:16.997186 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:13:16.997186 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:16.997186 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 177/359] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 30 16:13:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:16.998232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953affe-a95f-439e-af31-6b951b64701f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:16.998232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953affe-a95f-439e-af31-6b951b64701f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:16.998518 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3953affe-a95f-439e-af31-6b951b64701f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:17.000820 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 178/360] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:17.001562 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953affe-a95f-439e-af31-6b951b64701f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:17.016641 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3530a4-20bb-44ba-8688-40f720a7aa2a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:17.067432 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7dc2fd-456b-4fce-8ab2-cd862ce26177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:17.068270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7dc2fd-456b-4fce-8ab2-cd862ce26177 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:17.068623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7dc2fd-456b-4fce-8ab2-cd862ce26177 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:17.068911 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7dc2fd-456b-4fce-8ab2-cd862ce26177 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:17.070353 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7dc2fd-456b-4fce-8ab2-cd862ce26177 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:17.071120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7dc2fd-456b-4fce-8ab2-cd862ce26177 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:17.073327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7dc2fd-456b-4fce-8ab2-cd862ce26177 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:17.082915 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 183/361] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:13:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:17.083902 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7dc2fd-456b-4fce-8ab2-cd862ce26177 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:17.143126 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3953affe-a95f-439e-af31-6b951b64701f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:17.165958 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:17.188971 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6734d0f-1e12-4ea8-84f5-6a2c45f8d1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:17.189701 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6734d0f-1e12-4ea8-84f5-6a2c45f8d1d9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:17.189991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6734d0f-1e12-4ea8-84f5-6a2c45f8d1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:17.190267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6734d0f-1e12-4ea8-84f5-6a2c45f8d1d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:17.191112 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6734d0f-1e12-4ea8-84f5-6a2c45f8d1d9 None admin] RBAC: Adding query filter params (group.id=026420b86aa24c14807e7f9fce4ef606, role.id=9ab3e302ead54950b7c30103f1e7fda7, scope.project.id=195eb382a49146b6b27a5cdc500c1a05) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:17.191425 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6734d0f-1e12-4ea8-84f5-6a2c45f8d1d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:17.195201 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6734d0f-1e12-4ea8-84f5-6a2c45f8d1d9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:17.205482 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 179/362] 10.222.0.67 () {56 vars in 1376 bytes} [Thu Jun 30 16:13:17 2022] GET /identity/v3/role_assignments?group.id=026420b86aa24c14807e7f9fce4ef606&scope.project.id=195eb382a49146b6b27a5cdc500c1a05&role.id=9ab3e302ead54950b7c30103f1e7fda7 => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:17.206747 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d74bb-5e53-4853-85f0-bb25eff96dc6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:17.207035 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d74bb-5e53-4853-85f0-bb25eff96dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:17.207341 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d74bb-5e53-4853-85f0-bb25eff96dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:17.223432 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:17.228332 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-771d74bb-5e53-4853-85f0-bb25eff96dc6 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:17.229669 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-771d74bb-5e53-4853-85f0-bb25eff96dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:17.234285 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7dc2fd-456b-4fce-8ab2-cd862ce26177 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:17.287439 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff6ace-ea05-40ec-a449-1b026330a970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:17.288566 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff6ace-ea05-40ec-a449-1b026330a970 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:17.289104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff6ace-ea05-40ec-a449-1b026330a970 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:17.289388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff6ace-ea05-40ec-a449-1b026330a970 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:17.334620 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 180/363] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:17.336062 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d74bb-5e53-4853-85f0-bb25eff96dc6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:17.380818 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff6ace-ea05-40ec-a449-1b026330a970 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:17.383140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff6ace-ea05-40ec-a449-1b026330a970 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:17.484915 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d74bb-5e53-4853-85f0-bb25eff96dc6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:17.506019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:17.521213 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:17.532919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 184/364] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:13:17.534064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff6ace-ea05-40ec-a449-1b026330a970 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:17.535681 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6686a9a-516a-418a-b10d-a0dd92fd36c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:17.536364 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6686a9a-516a-418a-b10d-a0dd92fd36c3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:17.536647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6686a9a-516a-418a-b10d-a0dd92fd36c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:17.537204 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6686a9a-516a-418a-b10d-a0dd92fd36c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:17.537903 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6686a9a-516a-418a-b10d-a0dd92fd36c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:17.538727 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6686a9a-516a-418a-b10d-a0dd92fd36c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:17.540553 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6686a9a-516a-418a-b10d-a0dd92fd36c3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:17.552915 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 181/365] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:13:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:17.554546 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6686a9a-516a-418a-b10d-a0dd92fd36c3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:17.650833 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff6ace-ea05-40ec-a449-1b026330a970 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:17.669272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6686a9a-516a-418a-b10d-a0dd92fd36c3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:17.689968 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6f835-8731-4a1c-a607-2aa382de320f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:17.690861 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6f835-8731-4a1c-a607-2aa382de320f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:17.691147 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6f835-8731-4a1c-a607-2aa382de320f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:17.691425 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6f835-8731-4a1c-a607-2aa382de320f None admin] PATH_INFO: `/v3/users/f5d383dd1ece47deae3c9f504bc5afda` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:17.692679 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6f835-8731-4a1c-a607-2aa382de320f None admin] RBAC: Authorizing `identity:get_user(user_id=f5d383dd1ece47deae3c9f504bc5afda)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:17.694145 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6f835-8731-4a1c-a607-2aa382de320f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:17.696758 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 185/366] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:13:17 2022] GET /identity/v3/users/f5d383dd1ece47deae3c9f504bc5afda => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:17.697634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6f835-8731-4a1c-a607-2aa382de320f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:17.708075 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358bb3b3-1c80-44cc-b0e3-999105e839f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:17.708880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bb3b3-1c80-44cc-b0e3-999105e839f8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:17.709162 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bb3b3-1c80-44cc-b0e3-999105e839f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:17.709435 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bb3b3-1c80-44cc-b0e3-999105e839f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:17.724271 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358bb3b3-1c80-44cc-b0e3-999105e839f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:17.734030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-358bb3b3-1c80-44cc-b0e3-999105e839f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:13:17.734030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:17.734030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:17.734030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:17.734030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:17.734030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:17.734030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:17.734030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:17.734030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:17.734030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:17.734030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:17.734030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:17.734030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:17.734030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:17.734030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:17.734030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:17.734913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:17.734913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:17.734913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:17.734913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:17.734913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:17.734913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:17.734913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:17.734913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:17.734913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:17.734913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:17.734913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:17.734913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:17.734913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:17.734913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:17.734913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:17.734913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:17.735748 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:17.735748 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:17.735748 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:17.735748 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:17.735748 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:17.735748 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:17.735748 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:17.735748 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:17.735748 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:17.735748 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:17.735748 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:13:17.735748 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:17.735748 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 182/367] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:13:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:17.736899 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358bb3b3-1c80-44cc-b0e3-999105e839f8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:17.770807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:17.785014 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:17.838424 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6f835-8731-4a1c-a607-2aa382de320f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:17.868774 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358bb3b3-1c80-44cc-b0e3-999105e839f8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:17.886155 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed555e-c674-4edb-8776-7596359b6d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:17.887172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed555e-c674-4edb-8776-7596359b6d71 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:17.887513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed555e-c674-4edb-8776-7596359b6d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:17.887850 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed555e-c674-4edb-8776-7596359b6d71 None admin] PATH_INFO: `/v3/projects/e54d8744be3c4672b24fa2e19f024af0` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:17.889159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed555e-c674-4edb-8776-7596359b6d71 None admin] RBAC: Authorizing `identity:get_project(project_id=e54d8744be3c4672b24fa2e19f024af0)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:17.890622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed555e-c674-4edb-8776-7596359b6d71 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:17.892583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 186/368] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:13:17 2022] GET /identity/v3/projects/e54d8744be3c4672b24fa2e19f024af0 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:17.893220 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed555e-c674-4edb-8776-7596359b6d71 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:17.917047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf6f12-abbd-4e85-8a95-848e25356163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:17.918031 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf6f12-abbd-4e85-8a95-848e25356163 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:17.918350 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf6f12-abbd-4e85-8a95-848e25356163 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:17.918633 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caf6f12-abbd-4e85-8a95-848e25356163 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:17.997389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf6f12-abbd-4e85-8a95-848e25356163 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:17.998948 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7caf6f12-abbd-4e85-8a95-848e25356163 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:18.014481 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed555e-c674-4edb-8776-7596359b6d71 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:18.050394 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:18.053015 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:18.060335 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60e75f-ba13-4f8c-ae79-cb759aa2dcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:18.063459 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60e75f-ba13-4f8c-ae79-cb759aa2dcbb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:18.063459 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60e75f-ba13-4f8c-ae79-cb759aa2dcbb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:18.063459 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60e75f-ba13-4f8c-ae79-cb759aa2dcbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:18.063459 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60e75f-ba13-4f8c-ae79-cb759aa2dcbb None admin] RBAC: Adding query filter params (role.id=6e5a95d194ca42cab7a30baff4554fd3, user.id=f5d383dd1ece47deae3c9f504bc5afda, scope.system=all) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:18.063459 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60e75f-ba13-4f8c-ae79-cb759aa2dcbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:18.065317 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60e75f-ba13-4f8c-ae79-cb759aa2dcbb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:18.080522 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 187/369] 10.222.0.67 () {56 vars in 1308 bytes} [Thu Jun 30 16:13:17 2022] GET /identity/v3/role_assignments?user.id=f5d383dd1ece47deae3c9f504bc5afda&scope.system=all&role.id=6e5a95d194ca42cab7a30baff4554fd3 => generated 510 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:18.084769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60e75f-ba13-4f8c-ae79-cb759aa2dcbb None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:18.191576 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 183/370] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:13:18.193864 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf6f12-abbd-4e85-8a95-848e25356163 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:18.286526 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60e75f-ba13-4f8c-ae79-cb759aa2dcbb None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:18.363176 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfeebd-7cfd-4632-9964-4f33e44f643d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:18.363892 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfeebd-7cfd-4632-9964-4f33e44f643d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:18.364183 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfeebd-7cfd-4632-9964-4f33e44f643d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:18.364466 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfeebd-7cfd-4632-9964-4f33e44f643d None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:18.365228 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbfeebd-7cfd-4632-9964-4f33e44f643d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:18.365540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbfeebd-7cfd-4632-9964-4f33e44f643d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:18.368817 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbfeebd-7cfd-4632-9964-4f33e44f643d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:18.385970 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 188/371] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:18.387388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfeebd-7cfd-4632-9964-4f33e44f643d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:18.389444 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7caf6f12-abbd-4e85-8a95-848e25356163 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:18.438195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b69cf-8dd9-4697-b464-9314fa10bbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:18.439593 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b69cf-8dd9-4697-b464-9314fa10bbdd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:18.440125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b69cf-8dd9-4697-b464-9314fa10bbdd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:18.440676 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667b69cf-8dd9-4697-b464-9314fa10bbdd None admin] PATH_INFO: `/v3/groups/026420b86aa24c14807e7f9fce4ef606` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:18.442233 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b69cf-8dd9-4697-b464-9314fa10bbdd None admin] RBAC: Authorizing `identity:get_group(group_id=026420b86aa24c14807e7f9fce4ef606)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:18.444028 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667b69cf-8dd9-4697-b464-9314fa10bbdd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:18.446637 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 184/372] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 30 16:13:18 2022] GET /identity/v3/groups/026420b86aa24c14807e7f9fce4ef606 => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:18.447826 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b69cf-8dd9-4697-b464-9314fa10bbdd None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:18.450729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:18.453192 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:18.511335 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfeebd-7cfd-4632-9964-4f33e44f643d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:18.550055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8409e8-82f6-4c8c-a9cd-251c4b2b7241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:18.550949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8409e8-82f6-4c8c-a9cd-251c4b2b7241 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:18.551238 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8409e8-82f6-4c8c-a9cd-251c4b2b7241 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:18.551586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8409e8-82f6-4c8c-a9cd-251c4b2b7241 None admin] PATH_INFO: `/v3/projects/195eb382a49146b6b27a5cdc500c1a05` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:18.552821 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8409e8-82f6-4c8c-a9cd-251c4b2b7241 None admin] RBAC: Authorizing `identity:get_project(project_id=195eb382a49146b6b27a5cdc500c1a05)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:18.555156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8409e8-82f6-4c8c-a9cd-251c4b2b7241 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:18.557208 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 189/373] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:13:18 2022] GET /identity/v3/projects/195eb382a49146b6b27a5cdc500c1a05 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:18.558116 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8409e8-82f6-4c8c-a9cd-251c4b2b7241 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:18.562414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667b69cf-8dd9-4697-b464-9314fa10bbdd None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:18.604248 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3178b76-254b-4136-b1c6-46aafa343809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:18.605275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3178b76-254b-4136-b1c6-46aafa343809 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:18.605779 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3178b76-254b-4136-b1c6-46aafa343809 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:18.606257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3178b76-254b-4136-b1c6-46aafa343809 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:18.672629 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8409e8-82f6-4c8c-a9cd-251c4b2b7241 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:18.684726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3178b76-254b-4136-b1c6-46aafa343809 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:18.686209 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3178b76-254b-4136-b1c6-46aafa343809 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:18.688396 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:18.699206 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:18.721769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba6e7b-80e0-4e10-a8e8-5ad196e15ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:18.722491 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba6e7b-80e0-4e10-a8e8-5ad196e15ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:18.722780 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba6e7b-80e0-4e10-a8e8-5ad196e15ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:18.723057 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ba6e7b-80e0-4e10-a8e8-5ad196e15ae2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:18.724483 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba6e7b-80e0-4e10-a8e8-5ad196e15ae2 None admin] RBAC: Adding query filter params (group.id=026420b86aa24c14807e7f9fce4ef606, role.id=cfeec012f4ec48138c0e637e9a7fde61, scope.project.id=e54d8744be3c4672b24fa2e19f024af0) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:18.724483 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba6e7b-80e0-4e10-a8e8-5ad196e15ae2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:18.726296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ba6e7b-80e0-4e10-a8e8-5ad196e15ae2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:18.737185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 190/374] 10.222.0.67 () {56 vars in 1376 bytes} [Thu Jun 30 16:13:18 2022] GET /identity/v3/role_assignments?group.id=026420b86aa24c14807e7f9fce4ef606&scope.project.id=e54d8744be3c4672b24fa2e19f024af0&role.id=cfeec012f4ec48138c0e637e9a7fde61 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:18.741852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba6e7b-80e0-4e10-a8e8-5ad196e15ae2 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:18.831953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 185/375] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:13:18.833475 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6f23c-506c-4292-b120-5e03982b6a53 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:18.833876 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6f23c-506c-4292-b120-5e03982b6a53 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:18.834216 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6f23c-506c-4292-b120-5e03982b6a53 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:18.836681 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 186/376] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:18.843729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e455550-80e2-4af3-a630-953333c8f869 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:18.844101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e455550-80e2-4af3-a630-953333c8f869 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:18.844453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e455550-80e2-4af3-a630-953333c8f869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:18.869311 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8e455550-80e2-4af3-a630-953333c8f869 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:18.870916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8e455550-80e2-4af3-a630-953333c8f869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:18.879484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ba6e7b-80e0-4e10-a8e8-5ad196e15ae2 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:18.927987 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d024e-9ebe-4519-aeb1-af780a1e10cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:18.928922 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d024e-9ebe-4519-aeb1-af780a1e10cd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:18.929212 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d024e-9ebe-4519-aeb1-af780a1e10cd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:18.929491 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d024e-9ebe-4519-aeb1-af780a1e10cd None admin] PATH_INFO: `/v3/groups/026420b86aa24c14807e7f9fce4ef606` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:18.930793 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d024e-9ebe-4519-aeb1-af780a1e10cd None admin] RBAC: Authorizing `identity:get_group(group_id=026420b86aa24c14807e7f9fce4ef606)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:18.932319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d024e-9ebe-4519-aeb1-af780a1e10cd None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:18.934911 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 191/377] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 30 16:13:18 2022] GET /identity/v3/groups/026420b86aa24c14807e7f9fce4ef606 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:18.934911 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d024e-9ebe-4519-aeb1-af780a1e10cd None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:18.985711 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 187/378] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:18.986782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e455550-80e2-4af3-a630-953333c8f869 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:19.021415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:19.026677 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:19.090567 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d024e-9ebe-4519-aeb1-af780a1e10cd None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:19.119506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e455550-80e2-4af3-a630-953333c8f869 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:19.153383 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247964bb-a568-4888-b90f-41933d6e3fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:19.154622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247964bb-a568-4888-b90f-41933d6e3fbe None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:19.155136 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247964bb-a568-4888-b90f-41933d6e3fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:19.155674 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247964bb-a568-4888-b90f-41933d6e3fbe None admin] PATH_INFO: `/v3/projects/3967c884c6b647c2a8e60d73c62e7927` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:19.157849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247964bb-a568-4888-b90f-41933d6e3fbe None admin] RBAC: Authorizing `identity:get_project(project_id=3967c884c6b647c2a8e60d73c62e7927)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:19.159326 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247964bb-a568-4888-b90f-41933d6e3fbe None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:19.161267 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 192/379] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:13:18 2022] GET /identity/v3/projects/3967c884c6b647c2a8e60d73c62e7927 => generated 344 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:19.161850 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247964bb-a568-4888-b90f-41933d6e3fbe None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:19.175521 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085b62e-4c42-4e36-9dcf-5862872a5eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:19.176309 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085b62e-4c42-4e36-9dcf-5862872a5eee None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:19.176598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085b62e-4c42-4e36-9dcf-5862872a5eee None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:19.176880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085b62e-4c42-4e36-9dcf-5862872a5eee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:19.178355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a085b62e-4c42-4e36-9dcf-5862872a5eee None admin] RBAC: Adding query filter params (group.id=026420b86aa24c14807e7f9fce4ef606, role.id=cfeec012f4ec48138c0e637e9a7fde61, scope.project.id=195eb382a49146b6b27a5cdc500c1a05) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:19.178672 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a085b62e-4c42-4e36-9dcf-5862872a5eee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:19.181490 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a085b62e-4c42-4e36-9dcf-5862872a5eee None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:19.188133 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 188/380] 10.222.0.67 () {56 vars in 1376 bytes} [Thu Jun 30 16:13:18 2022] GET /identity/v3/role_assignments?group.id=026420b86aa24c14807e7f9fce4ef606&scope.project.id=195eb382a49146b6b27a5cdc500c1a05&role.id=cfeec012f4ec48138c0e637e9a7fde61 => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:19.189171 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085b62e-4c42-4e36-9dcf-5862872a5eee None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:19.293244 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247964bb-a568-4888-b90f-41933d6e3fbe None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:19.311850 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:19.324381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:19.331693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085b62e-4c42-4e36-9dcf-5862872a5eee None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:19.332214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ad71c-e542-454d-94af-711ff2a21b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:19.333099 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ad71c-e542-454d-94af-711ff2a21b4e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:19.333385 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ad71c-e542-454d-94af-711ff2a21b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:19.333663 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5ad71c-e542-454d-94af-711ff2a21b4e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:19.347305 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5ad71c-e542-454d-94af-711ff2a21b4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:19.356427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-2b5ad71c-e542-454d-94af-711ff2a21b4e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:13:19.356427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:19.356427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:19.356427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:19.356427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:19.356427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:19.356427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:19.356427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:19.356427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:19.356427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:19.356427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:19.356427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:19.356427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:19.356427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:19.356427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:19.356427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:19.358329 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:19.358329 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:19.358329 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:19.358329 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:19.358329 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:19.358329 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:19.358329 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:19.358329 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:19.358329 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:19.358329 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:19.358329 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:19.358329 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:19.358329 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:19.358329 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:19.358329 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:19.358329 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:19.360426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:19.360426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:19.360426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:19.360426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:19.360426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:19.360426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:19.360426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:19.360426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:19.360426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:19.360426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:19.360426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:13:19.360426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:19.360426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 193/381] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:19.364789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ad71c-e542-454d-94af-711ff2a21b4e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:19.390996 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ad06f-b8c3-4541-8bc5-dd06b2dd1812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:19.391912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ad06f-b8c3-4541-8bc5-dd06b2dd1812 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:19.392189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ad06f-b8c3-4541-8bc5-dd06b2dd1812 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:19.392442 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ad06f-b8c3-4541-8bc5-dd06b2dd1812 None admin] PATH_INFO: `/v3/groups/79ce9e920d8b4f3cbbba85006e4c647b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:19.407547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ad06f-b8c3-4541-8bc5-dd06b2dd1812 None admin] RBAC: Authorizing `identity:get_group(group_id=79ce9e920d8b4f3cbbba85006e4c647b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:19.409048 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4ad06f-b8c3-4541-8bc5-dd06b2dd1812 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:19.415490 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 189/382] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 30 16:13:19 2022] GET /identity/v3/groups/79ce9e920d8b4f3cbbba85006e4c647b => generated 229 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:19.425472 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ad06f-b8c3-4541-8bc5-dd06b2dd1812 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:19.548609 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5ad71c-e542-454d-94af-711ff2a21b4e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:19.594542 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8252a9c-66bc-49a0-b666-f8eafbb8e2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:19.595213 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8252a9c-66bc-49a0-b666-f8eafbb8e2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:19.595671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8252a9c-66bc-49a0-b666-f8eafbb8e2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:19.595900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8252a9c-66bc-49a0-b666-f8eafbb8e2b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:19.596512 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8252a9c-66bc-49a0-b666-f8eafbb8e2b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:19.596820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8252a9c-66bc-49a0-b666-f8eafbb8e2b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:19.598216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8252a9c-66bc-49a0-b666-f8eafbb8e2b7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:19.613552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ad06f-b8c3-4541-8bc5-dd06b2dd1812 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:19.615794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 194/383] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:13:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:19.621342 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8252a9c-66bc-49a0-b666-f8eafbb8e2b7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:19.665237 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5d430-3ef1-4e01-8f0e-c67d5bb8a392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:19.666101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5d430-3ef1-4e01-8f0e-c67d5bb8a392 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:19.666401 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5d430-3ef1-4e01-8f0e-c67d5bb8a392 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:19.666798 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5d430-3ef1-4e01-8f0e-c67d5bb8a392 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:19.667569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5d430-3ef1-4e01-8f0e-c67d5bb8a392 None admin] RBAC: Adding query filter params (group.id=79ce9e920d8b4f3cbbba85006e4c647b, role.id=8acc34a163874a60b03728b814eaf075, scope.project.id=3967c884c6b647c2a8e60d73c62e7927) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:19.667887 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5d430-3ef1-4e01-8f0e-c67d5bb8a392 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:19.669420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5d430-3ef1-4e01-8f0e-c67d5bb8a392 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:19.677033 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:19.681946 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 190/384] 10.222.0.67 () {56 vars in 1376 bytes} [Thu Jun 30 16:13:19 2022] GET /identity/v3/role_assignments?group.id=79ce9e920d8b4f3cbbba85006e4c647b&scope.project.id=3967c884c6b647c2a8e60d73c62e7927&role.id=8acc34a163874a60b03728b814eaf075 => generated 260 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:19.748503 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8252a9c-66bc-49a0-b666-f8eafbb8e2b7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:19.791165 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2795763-1761-4031-8721-606ec2d58703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:19.792084 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2795763-1761-4031-8721-606ec2d58703 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:19.792362 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2795763-1761-4031-8721-606ec2d58703 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:19.792634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2795763-1761-4031-8721-606ec2d58703 None admin] PATH_INFO: `/v3/groups/026420b86aa24c14807e7f9fce4ef606` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:19.796385 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2795763-1761-4031-8721-606ec2d58703 None admin] RBAC: Authorizing `identity:get_group(group_id=026420b86aa24c14807e7f9fce4ef606)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:19.797904 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2795763-1761-4031-8721-606ec2d58703 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:19.800041 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 195/385] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 30 16:13:19 2022] GET /identity/v3/groups/026420b86aa24c14807e7f9fce4ef606 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:19.808082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5d430-3ef1-4e01-8f0e-c67d5bb8a392 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:19.868400 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:19.948798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85fb37-4c84-49ec-9a71-df95e98aa938 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:19.949379 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85fb37-4c84-49ec-9a71-df95e98aa938 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:19.949656 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85fb37-4c84-49ec-9a71-df95e98aa938 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:19.951206 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 196/386] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:13:19.956642 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598da30-60e6-45e0-9daf-78b79717a40f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:19.956926 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598da30-60e6-45e0-9daf-78b79717a40f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:19.957204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598da30-60e6-45e0-9daf-78b79717a40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:19.973956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5d430-3ef1-4e01-8f0e-c67d5bb8a392 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:19.978657 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d598da30-60e6-45e0-9daf-78b79717a40f None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:19.981526 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d598da30-60e6-45e0-9daf-78b79717a40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:20.048869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2721497-6c0f-4480-ac36-da004eb00258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:20.050097 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2721497-6c0f-4480-ac36-da004eb00258 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:20.050326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2721497-6c0f-4480-ac36-da004eb00258 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:20.050602 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2721497-6c0f-4480-ac36-da004eb00258 None admin] PATH_INFO: `/v3/projects/e54d8744be3c4672b24fa2e19f024af0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:20.055592 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2721497-6c0f-4480-ac36-da004eb00258 None admin] RBAC: Authorizing `identity:get_project(project_id=e54d8744be3c4672b24fa2e19f024af0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:20.057363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2721497-6c0f-4480-ac36-da004eb00258 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:20.059062 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 191/387] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:13:19 2022] GET /identity/v3/projects/e54d8744be3c4672b24fa2e19f024af0 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:20.064648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2721497-6c0f-4480-ac36-da004eb00258 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:20.073103 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 197/388] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:13:20.098895 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d598da30-60e6-45e0-9daf-78b79717a40f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:20.143171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:20.219632 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:20.231543 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2721497-6c0f-4480-ac36-da004eb00258 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:20.289074 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d598da30-60e6-45e0-9daf-78b79717a40f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:20.298622 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6e096-0d34-4934-84f1-8befede3edc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:20.299540 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6e096-0d34-4934-84f1-8befede3edc9 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:20.299861 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6e096-0d34-4934-84f1-8befede3edc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:20.300192 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff6e096-0d34-4934-84f1-8befede3edc9 None admin] PATH_INFO: `/v3/projects/e54d8744be3c4672b24fa2e19f024af0/groups/026420b86aa24c14807e7f9fce4ef606/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:20.305351 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff6e096-0d34-4934-84f1-8befede3edc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=e54d8744be3c4672b24fa2e19f024af0, group_id=026420b86aa24c14807e7f9fce4ef606, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:20.306777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff6e096-0d34-4934-84f1-8befede3edc9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:20.339505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e42100-04ec-4197-9bcb-2f728c02078d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:20.340365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e42100-04ec-4197-9bcb-2f728c02078d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:20.341536 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e42100-04ec-4197-9bcb-2f728c02078d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:20.341825 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e42100-04ec-4197-9bcb-2f728c02078d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:20.356450 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e42100-04ec-4197-9bcb-2f728c02078d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:20.364971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-c9e42100-04ec-4197-9bcb-2f728c02078d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:13:20.364971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:20.364971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:20.364971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:20.364971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:20.364971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:20.364971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:20.364971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:20.364971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:20.364971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:20.364971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:20.364971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:20.364971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:20.364971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:20.364971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:20.364971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:20.365856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:20.365856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:20.365856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:20.365856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:20.365856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:20.365856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:20.365856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:20.365856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:20.365856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:20.365856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:20.365856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:20.365856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:20.365856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:20.365856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:20.365856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:20.365856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:20.367203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:20.367203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:20.367203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:20.367203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:20.367203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:20.367203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:20.367203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:20.367203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:20.367203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:20.367203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:20.367203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:13:20.367203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:20.369920 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 198/389] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:20.375960 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e42100-04ec-4197-9bcb-2f728c02078d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:20.421848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 192/390] 10.222.0.67 () {58 vars in 1442 bytes} [Thu Jun 30 16:13:20 2022] PUT /identity/v3/projects/e54d8744be3c4672b24fa2e19f024af0/groups/026420b86aa24c14807e7f9fce4ef606/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:13:20.483434 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:20.495017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e42100-04ec-4197-9bcb-2f728c02078d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:20.545718 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46843ef0-631b-4c78-8c7b-73d4afd24607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:20.546397 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46843ef0-631b-4c78-8c7b-73d4afd24607 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:20.546682 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46843ef0-631b-4c78-8c7b-73d4afd24607 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:20.546963 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46843ef0-631b-4c78-8c7b-73d4afd24607 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:20.547747 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46843ef0-631b-4c78-8c7b-73d4afd24607 None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:20.548060 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46843ef0-631b-4c78-8c7b-73d4afd24607 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:20.549495 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46843ef0-631b-4c78-8c7b-73d4afd24607 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:20.560453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 199/391] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:13:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:20.566165 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6e096-0d34-4934-84f1-8befede3edc9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:20.680838 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff6e096-0d34-4934-84f1-8befede3edc9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:20.727112 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00875d4-310c-4dfa-90a5-f0d5cfe92dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:20.728098 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00875d4-310c-4dfa-90a5-f0d5cfe92dfb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:20.728388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00875d4-310c-4dfa-90a5-f0d5cfe92dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:20.728700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00875d4-310c-4dfa-90a5-f0d5cfe92dfb None admin] PATH_INFO: `/v3/groups/026420b86aa24c14807e7f9fce4ef606` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:20.729953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00875d4-310c-4dfa-90a5-f0d5cfe92dfb None admin] RBAC: Authorizing `identity:get_group(group_id=026420b86aa24c14807e7f9fce4ef606)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:20.731425 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00875d4-310c-4dfa-90a5-f0d5cfe92dfb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:20.733340 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 193/392] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 30 16:13:20 2022] GET /identity/v3/groups/026420b86aa24c14807e7f9fce4ef606 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:20.738874 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46843ef0-631b-4c78-8c7b-73d4afd24607 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:20.849690 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46843ef0-631b-4c78-8c7b-73d4afd24607 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:20.887445 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119090c-58e5-412e-9515-f99aa5c39fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:20.888340 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119090c-58e5-412e-9515-f99aa5c39fcc None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:20.888623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119090c-58e5-412e-9515-f99aa5c39fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:20.888896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119090c-58e5-412e-9515-f99aa5c39fcc None admin] PATH_INFO: `/v3/projects/195eb382a49146b6b27a5cdc500c1a05` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:20.890101 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119090c-58e5-412e-9515-f99aa5c39fcc None admin] RBAC: Authorizing `identity:get_project(project_id=195eb382a49146b6b27a5cdc500c1a05)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:20.891617 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119090c-58e5-412e-9515-f99aa5c39fcc None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:20.893904 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 200/393] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:13:20 2022] GET /identity/v3/projects/195eb382a49146b6b27a5cdc500c1a05 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:20.900551 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00875d4-310c-4dfa-90a5-f0d5cfe92dfb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:20.931286 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:21.040277 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00875d4-310c-4dfa-90a5-f0d5cfe92dfb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:21.095590 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d827c1d-e7bb-4c9e-8f24-85c7e1d307c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:21.096482 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d827c1d-e7bb-4c9e-8f24-85c7e1d307c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:21.096761 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d827c1d-e7bb-4c9e-8f24-85c7e1d307c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:21.097034 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d827c1d-e7bb-4c9e-8f24-85c7e1d307c3 None admin] PATH_INFO: `/v3/projects/195eb382a49146b6b27a5cdc500c1a05/groups/026420b86aa24c14807e7f9fce4ef606/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:21.099104 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d827c1d-e7bb-4c9e-8f24-85c7e1d307c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=195eb382a49146b6b27a5cdc500c1a05, group_id=026420b86aa24c14807e7f9fce4ef606, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:21.101223 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d827c1d-e7bb-4c9e-8f24-85c7e1d307c3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:21.187671 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 194/394] 10.222.0.67 () {58 vars in 1442 bytes} [Thu Jun 30 16:13:20 2022] PUT /identity/v3/projects/195eb382a49146b6b27a5cdc500c1a05/groups/026420b86aa24c14807e7f9fce4ef606/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:13:21.695234 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79d682-a924-41e2-909a-8f7444e6b9da None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:21.695660 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79d682-a924-41e2-909a-8f7444e6b9da None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:21.695840 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79d682-a924-41e2-909a-8f7444e6b9da None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:21.697490 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 201/395] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:21.703090 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a364b5-5873-4b5f-bbb8-4bf638abdb0f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:21.703381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a364b5-5873-4b5f-bbb8-4bf638abdb0f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:21.703844 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a364b5-5873-4b5f-bbb8-4bf638abdb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:21.728204 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-76a364b5-5873-4b5f-bbb8-4bf638abdb0f None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:21.729516 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-76a364b5-5873-4b5f-bbb8-4bf638abdb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:21.822073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 195/396] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:21.857670 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79d682-a924-41e2-909a-8f7444e6b9da None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:21.894721 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:22.000342 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79d682-a924-41e2-909a-8f7444e6b9da None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:22.020610 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562edc4-5940-4734-a647-cf32c142a13e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:22.021112 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562edc4-5940-4734-a647-cf32c142a13e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:22.021219 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562edc4-5940-4734-a647-cf32c142a13e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:22.023604 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 196/397] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:13:22.032144 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72c4f0-26f9-427a-a2d3-e952a0c22abd None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:22.032766 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72c4f0-26f9-427a-a2d3-e952a0c22abd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:22.033247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72c4f0-26f9-427a-a2d3-e952a0c22abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:22.058162 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5a72c4f0-26f9-427a-a2d3-e952a0c22abd None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:22.061175 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5a72c4f0-26f9-427a-a2d3-e952a0c22abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:22.068656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:22.073348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e55013-070c-4dcb-b8e2-f550bb9f3c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:22.074521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e55013-070c-4dcb-b8e2-f550bb9f3c65 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:22.075051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e55013-070c-4dcb-b8e2-f550bb9f3c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:22.075556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e55013-070c-4dcb-b8e2-f550bb9f3c65 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:22.098695 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e55013-070c-4dcb-b8e2-f550bb9f3c65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:22.114777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-d0e55013-070c-4dcb-b8e2-f550bb9f3c65 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:13:22.114777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:22.114777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:22.114777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:22.114777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:22.114777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:22.114777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:22.114777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:22.114777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:22.114777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:22.114777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:22.114777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:22.114777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:22.114777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:22.114777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:22.114777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:22.115784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:22.115784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:22.115784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:22.115784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:22.115784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:22.115784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:22.115784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:22.115784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:22.115784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:22.115784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:22.115784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:22.115784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:22.115784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:22.115784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:22.115784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:22.115784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:22.116705 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:22.116705 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:22.116705 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:22.116705 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:22.116705 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:22.116705 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:22.116705 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:22.116705 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:22.116705 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:22.116705 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:22.116705 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:13:22.116705 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:22.125916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 202/398] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 30 16:13:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:22.130634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e55013-070c-4dcb-b8e2-f550bb9f3c65 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:22.215812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 197/399] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:13:22.258989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:22.269098 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72c4f0-26f9-427a-a2d3-e952a0c22abd None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:22.286417 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e55013-070c-4dcb-b8e2-f550bb9f3c65 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:22.347558 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c8465-4fef-4ac8-a422-f3b39d1921ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:22.348639 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c8465-4fef-4ac8-a422-f3b39d1921ba None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:22.349173 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c8465-4fef-4ac8-a422-f3b39d1921ba None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:22.349669 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c8465-4fef-4ac8-a422-f3b39d1921ba None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:22.350777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c8465-4fef-4ac8-a422-f3b39d1921ba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:22.351330 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c8465-4fef-4ac8-a422-f3b39d1921ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:22.352962 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c8465-4fef-4ac8-a422-f3b39d1921ba None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:22.362243 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 203/400] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:13:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:22.368562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c8465-4fef-4ac8-a422-f3b39d1921ba None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:22.432094 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72c4f0-26f9-427a-a2d3-e952a0c22abd None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:22.501117 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a53d8e-4451-4fa0-9cfa-de8c28965e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:22.501948 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a53d8e-4451-4fa0-9cfa-de8c28965e46 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:22.502232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a53d8e-4451-4fa0-9cfa-de8c28965e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:22.502512 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a53d8e-4451-4fa0-9cfa-de8c28965e46 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:22.519995 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:22.527518 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a53d8e-4451-4fa0-9cfa-de8c28965e46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:22.538931 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c8465-4fef-4ac8-a422-f3b39d1921ba None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:22.547463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-89a53d8e-4451-4fa0-9cfa-de8c28965e46 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:13:22.547463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:22.547463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:22.547463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:22.547463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:22.547463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:22.547463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:22.547463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:22.547463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:22.547463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:22.547463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:22.547463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:22.547463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:22.547463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:22.547463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:22.547463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:22.548388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:22.548388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:22.548388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:22.548388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:22.548388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:22.548388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:22.548388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:22.548388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:22.548388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:22.548388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:22.548388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:22.548388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:22.548388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:22.548388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:22.548388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:22.548388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:22.549157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:22.549157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:22.549157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:22.549157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:22.549157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:22.549157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:22.549157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:22.549157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:22.549157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:22.549157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:22.549157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:13:22.549157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:22.555932 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 198/401] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 30 16:13:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:13:22.564511 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a53d8e-4451-4fa0-9cfa-de8c28965e46 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:22.585519 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa31f4b-6274-4141-9c91-de0cadac678a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:22.586816 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa31f4b-6274-4141-9c91-de0cadac678a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:22.587323 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa31f4b-6274-4141-9c91-de0cadac678a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:22.587833 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa31f4b-6274-4141-9c91-de0cadac678a None admin] PATH_INFO: `/v3/groups/026420b86aa24c14807e7f9fce4ef606` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:22.590495 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa31f4b-6274-4141-9c91-de0cadac678a None admin] RBAC: Authorizing `identity:get_group(group_id=026420b86aa24c14807e7f9fce4ef606)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:22.592203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa31f4b-6274-4141-9c91-de0cadac678a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:22.595047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 204/402] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 30 16:13:22 2022] GET /identity/v3/groups/026420b86aa24c14807e7f9fce4ef606 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:22.603439 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa31f4b-6274-4141-9c91-de0cadac678a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:22.640155 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:22.719669 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a53d8e-4451-4fa0-9cfa-de8c28965e46 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:22.758317 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02baefc7-d577-4234-a658-07cc769f752d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:22.759677 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02baefc7-d577-4234-a658-07cc769f752d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:22.760149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02baefc7-d577-4234-a658-07cc769f752d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:22.761051 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02baefc7-d577-4234-a658-07cc769f752d None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:22.762662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02baefc7-d577-4234-a658-07cc769f752d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:22.763187 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02baefc7-d577-4234-a658-07cc769f752d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:22.764843 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa31f4b-6274-4141-9c91-de0cadac678a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:22.765588 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02baefc7-d577-4234-a658-07cc769f752d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:22.782879 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 199/403] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:13:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:22.788936 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076d22d-bc52-4c9a-86e7-c9ba410328e7 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:22.789229 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076d22d-bc52-4c9a-86e7-c9ba410328e7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:22.789509 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7076d22d-bc52-4c9a-86e7-c9ba410328e7 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:22.791101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 200/404] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:22.791857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076d22d-bc52-4c9a-86e7-c9ba410328e7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:22.814544 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0fae91-b516-4bda-9c8f-7226b8835362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:22.815416 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fae91-b516-4bda-9c8f-7226b8835362 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:22.815730 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fae91-b516-4bda-9c8f-7226b8835362 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:22.816007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fae91-b516-4bda-9c8f-7226b8835362 None admin] PATH_INFO: `/v3/projects/e54d8744be3c4672b24fa2e19f024af0` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:22.817199 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0fae91-b516-4bda-9c8f-7226b8835362 None admin] RBAC: Authorizing `identity:get_project(project_id=e54d8744be3c4672b24fa2e19f024af0)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:22.818681 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0fae91-b516-4bda-9c8f-7226b8835362 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:22.821456 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 205/405] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:13:22 2022] GET /identity/v3/projects/e54d8744be3c4672b24fa2e19f024af0 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:22.822747 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9f23f-5d09-4d55-a260-95be9c27b224 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:22.823030 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9f23f-5d09-4d55-a260-95be9c27b224 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:22.823308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9f23f-5d09-4d55-a260-95be9c27b224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:22.840927 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b0d9f23f-5d09-4d55-a260-95be9c27b224 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:22.842135 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b0d9f23f-5d09-4d55-a260-95be9c27b224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:22.864112 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:22.915275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7076d22d-bc52-4c9a-86e7-c9ba410328e7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:22.923897 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 206/406] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:22.931500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9f23f-5d09-4d55-a260-95be9c27b224 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:22.940469 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:22.980350 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d1f7f-779f-4f9e-ae2e-f036fe8a1f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:22.981466 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d1f7f-779f-4f9e-ae2e-f036fe8a1f11 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:22.981894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d1f7f-779f-4f9e-ae2e-f036fe8a1f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:22.982342 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d1f7f-779f-4f9e-ae2e-f036fe8a1f11 None admin] PATH_INFO: `/v3/groups/026420b86aa24c14807e7f9fce4ef606` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:22.983772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d1f7f-779f-4f9e-ae2e-f036fe8a1f11 None admin] RBAC: Authorizing `identity:get_group(group_id=026420b86aa24c14807e7f9fce4ef606)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:22.985365 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d1f7f-779f-4f9e-ae2e-f036fe8a1f11 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:22.987499 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 201/407] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 30 16:13:22 2022] GET /identity/v3/groups/026420b86aa24c14807e7f9fce4ef606 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:22.988382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d1f7f-779f-4f9e-ae2e-f036fe8a1f11 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:23.045367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9f23f-5d09-4d55-a260-95be9c27b224 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:23.093229 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd09f6-7058-4506-9c8a-6231b3c3ecb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:23.094700 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd09f6-7058-4506-9c8a-6231b3c3ecb4 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:23.094988 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd09f6-7058-4506-9c8a-6231b3c3ecb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:23.095265 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd09f6-7058-4506-9c8a-6231b3c3ecb4 None admin] PATH_INFO: `/v3/projects/e54d8744be3c4672b24fa2e19f024af0/groups/026420b86aa24c14807e7f9fce4ef606/roles/cfeec012f4ec48138c0e637e9a7fde61` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:23.098337 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd09f6-7058-4506-9c8a-6231b3c3ecb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e54d8744be3c4672b24fa2e19f024af0, group_id=026420b86aa24c14807e7f9fce4ef606, role_id=cfeec012f4ec48138c0e637e9a7fde61)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:23.100009 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd09f6-7058-4506-9c8a-6231b3c3ecb4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:23.117171 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d1f7f-779f-4f9e-ae2e-f036fe8a1f11 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:23.147418 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:23.153815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bdb4f5-7ffa-4532-8b15-f88ee1329669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:23.154734 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bdb4f5-7ffa-4532-8b15-f88ee1329669 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:23.155016 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bdb4f5-7ffa-4532-8b15-f88ee1329669 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:23.155285 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bdb4f5-7ffa-4532-8b15-f88ee1329669 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:23.166159 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bdb4f5-7ffa-4532-8b15-f88ee1329669 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:23.174107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-81bdb4f5-7ffa-4532-8b15-f88ee1329669 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:13:23.174107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:23.174107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:23.174107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:23.174107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:23.174107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:23.174107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:23.174107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:23.174107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:23.174107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:23.174107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:23.174107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:23.174107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:23.174107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:23.174107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:23.174107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:23.174998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:23.174998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:23.174998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:23.174998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:23.174998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:23.174998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:23.174998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:23.174998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:23.174998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:23.174998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:23.174998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:23.174998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:23.174998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:23.174998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:23.174998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:23.174998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:23.176424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:23.176424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:23.176424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:23.176424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:23.176424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:23.176424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:23.176424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:23.176424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:23.176424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:23.176424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:23.176424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:13:23.176424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:23.176424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 202/408] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:13:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:23.176424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bdb4f5-7ffa-4532-8b15-f88ee1329669 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:23.200334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 207/409] 10.222.0.67 () {58 vars in 1442 bytes} [Thu Jun 30 16:13:22 2022] PUT /identity/v3/projects/e54d8744be3c4672b24fa2e19f024af0/groups/026420b86aa24c14807e7f9fce4ef606/roles/cfeec012f4ec48138c0e637e9a7fde61 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:13:23.201006 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd09f6-7058-4506-9c8a-6231b3c3ecb4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:23.297184 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:23.326702 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd09f6-7058-4506-9c8a-6231b3c3ecb4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:23.328881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bdb4f5-7ffa-4532-8b15-f88ee1329669 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:23.367302 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e89ab9-76a0-43ae-8104-6adea52e9282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:23.368020 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e89ab9-76a0-43ae-8104-6adea52e9282 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:23.368700 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e89ab9-76a0-43ae-8104-6adea52e9282 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:23.368700 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e89ab9-76a0-43ae-8104-6adea52e9282 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:23.369344 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e89ab9-76a0-43ae-8104-6adea52e9282 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:23.369647 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e89ab9-76a0-43ae-8104-6adea52e9282 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:23.371551 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e89ab9-76a0-43ae-8104-6adea52e9282 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:23.380173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de845b-a893-4244-a10e-682c77de6d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:23.381205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de845b-a893-4244-a10e-682c77de6d9c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:23.382382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de845b-a893-4244-a10e-682c77de6d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:23.382717 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de845b-a893-4244-a10e-682c77de6d9c None admin] PATH_INFO: `/v3/projects/195eb382a49146b6b27a5cdc500c1a05` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:23.384101 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 208/410] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:23.384213 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de845b-a893-4244-a10e-682c77de6d9c None admin] RBAC: Authorizing `identity:get_project(project_id=195eb382a49146b6b27a5cdc500c1a05)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:23.385269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41710f-72ff-4cce-92ee-3b22e7018025 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:23.385597 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de845b-a893-4244-a10e-682c77de6d9c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:23.385701 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41710f-72ff-4cce-92ee-3b22e7018025 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:23.385883 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41710f-72ff-4cce-92ee-3b22e7018025 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:23.387573 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 209/412] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:23.392053 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 203/412] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:13:23 2022] GET /identity/v3/projects/195eb382a49146b6b27a5cdc500c1a05 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:23.392363 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41710f-72ff-4cce-92ee-3b22e7018025 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:23.394271 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de845b-a893-4244-a10e-682c77de6d9c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:23.486932 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:23.509484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41710f-72ff-4cce-92ee-3b22e7018025 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:23.547867 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de845b-a893-4244-a10e-682c77de6d9c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:23.548795 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8147e8a9-b3a5-4fe5-b9cd-d8f3c64649df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:23.549829 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147e8a9-b3a5-4fe5-b9cd-d8f3c64649df None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:23.550197 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147e8a9-b3a5-4fe5-b9cd-d8f3c64649df None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:23.550611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8147e8a9-b3a5-4fe5-b9cd-d8f3c64649df None admin] PATH_INFO: `/v3/groups/79ce9e920d8b4f3cbbba85006e4c647b` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:23.555797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8147e8a9-b3a5-4fe5-b9cd-d8f3c64649df None admin] RBAC: Authorizing `identity:get_group(group_id=79ce9e920d8b4f3cbbba85006e4c647b)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:23.558185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8147e8a9-b3a5-4fe5-b9cd-d8f3c64649df None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:23.565325 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 210/413] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 30 16:13:23 2022] GET /identity/v3/groups/79ce9e920d8b4f3cbbba85006e4c647b => generated 229 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:23.565325 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c712f53-89e8-4b16-9e6c-1e6dbbdfeb87 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:23.565325 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c712f53-89e8-4b16-9e6c-1e6dbbdfeb87 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:23.566384 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c712f53-89e8-4b16-9e6c-1e6dbbdfeb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:23.575673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:23.592767 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7c712f53-89e8-4b16-9e6c-1e6dbbdfeb87 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:23.593067 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b500e6b-7b53-4096-803f-a6d356b46c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:23.593931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b500e6b-7b53-4096-803f-a6d356b46c99 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:23.594307 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b500e6b-7b53-4096-803f-a6d356b46c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:23.594580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b500e6b-7b53-4096-803f-a6d356b46c99 None admin] PATH_INFO: `/v3/projects/195eb382a49146b6b27a5cdc500c1a05/groups/026420b86aa24c14807e7f9fce4ef606/roles/cfeec012f4ec48138c0e637e9a7fde61` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:23.595328 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7c712f53-89e8-4b16-9e6c-1e6dbbdfeb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:23.596990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b500e6b-7b53-4096-803f-a6d356b46c99 None admin] RBAC: Authorizing `identity:create_grant(project_id=195eb382a49146b6b27a5cdc500c1a05, group_id=026420b86aa24c14807e7f9fce4ef606, role_id=cfeec012f4ec48138c0e637e9a7fde61)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:23.598500 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b500e6b-7b53-4096-803f-a6d356b46c99 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:23.671779 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 204/414] 10.222.0.67 () {58 vars in 1442 bytes} [Thu Jun 30 16:13:23 2022] PUT /identity/v3/projects/195eb382a49146b6b27a5cdc500c1a05/groups/026420b86aa24c14807e7f9fce4ef606/roles/cfeec012f4ec48138c0e637e9a7fde61 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:13:23.674349 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b500e6b-7b53-4096-803f-a6d356b46c99 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:23.684168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 211/415] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:23.742553 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c712f53-89e8-4b16-9e6c-1e6dbbdfeb87 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:23.823441 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b500e6b-7b53-4096-803f-a6d356b46c99 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:23.846042 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:23.867226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c9d0a-02f0-4839-b0a9-65de9d953399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:23.869790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c9d0a-02f0-4839-b0a9-65de9d953399 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:23.871623 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c9d0a-02f0-4839-b0a9-65de9d953399 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:23.872349 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c9d0a-02f0-4839-b0a9-65de9d953399 None admin] PATH_INFO: `/v3/projects/3967c884c6b647c2a8e60d73c62e7927` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:23.873882 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c9d0a-02f0-4839-b0a9-65de9d953399 None admin] RBAC: Authorizing `identity:get_project(project_id=3967c884c6b647c2a8e60d73c62e7927)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:23.876585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731c9d0a-02f0-4839-b0a9-65de9d953399 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:23.886687 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 205/416] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:13:23 2022] GET /identity/v3/projects/3967c884c6b647c2a8e60d73c62e7927 => generated 344 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:23.889295 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c9d0a-02f0-4839-b0a9-65de9d953399 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:23.949814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c712f53-89e8-4b16-9e6c-1e6dbbdfeb87 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:23.988607 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0644997-b982-49d0-b2d5-6a6934150702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:23.989902 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0644997-b982-49d0-b2d5-6a6934150702 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:23.990491 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0644997-b982-49d0-b2d5-6a6934150702 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:23.991510 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0644997-b982-49d0-b2d5-6a6934150702 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:23.993827 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:24.005190 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0644997-b982-49d0-b2d5-6a6934150702 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:24.014601 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-e0644997-b982-49d0-b2d5-6a6934150702 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:13:24.014601 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:24.014601 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:24.014601 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:24.014601 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:24.014601 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:24.014601 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:24.014601 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:24.014601 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:24.014601 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:24.014601 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:24.014601 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:24.014601 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:24.014601 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:24.014601 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:24.014601 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:24.015556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:24.015556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:24.015556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:24.015556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:24.015556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:24.015556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:24.015556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:24.015556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:24.015556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:24.015556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:24.015556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:24.015556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:24.015556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:24.015556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:24.015556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:24.015556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:24.016352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:24.016352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:24.016352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:24.016352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:24.016352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:24.016352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:24.016352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:24.016352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:24.016352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:24.016352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:24.016352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:13:24.016352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:24.019226 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 212/417] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:13:24.019337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c9d0a-02f0-4839-b0a9-65de9d953399 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:24.021610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a54e7-2e7b-4b26-a1df-ec7f3af2461c None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:24.022454 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a54e7-2e7b-4b26-a1df-ec7f3af2461c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:24.023136 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a54e7-2e7b-4b26-a1df-ec7f3af2461c None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:24.025282 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 213/418] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:24.026187 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a54e7-2e7b-4b26-a1df-ec7f3af2461c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:24.060640 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a104d191-9491-4541-920a-abf9c1b8609f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:24.061530 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104d191-9491-4541-920a-abf9c1b8609f None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:24.061813 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104d191-9491-4541-920a-abf9c1b8609f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:24.062106 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a104d191-9491-4541-920a-abf9c1b8609f None admin] PATH_INFO: `/v3/projects/3967c884c6b647c2a8e60d73c62e7927/groups/79ce9e920d8b4f3cbbba85006e4c647b/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:24.064432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104d191-9491-4541-920a-abf9c1b8609f None admin] RBAC: Authorizing `identity:create_grant(project_id=3967c884c6b647c2a8e60d73c62e7927, group_id=79ce9e920d8b4f3cbbba85006e4c647b, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:24.066470 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a104d191-9491-4541-920a-abf9c1b8609f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:24.140267 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a54e7-2e7b-4b26-a1df-ec7f3af2461c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:24.159709 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 206/419] 10.222.0.67 () {58 vars in 1442 bytes} [Thu Jun 30 16:13:23 2022] PUT /identity/v3/projects/3967c884c6b647c2a8e60d73c62e7927/groups/79ce9e920d8b4f3cbbba85006e4c647b/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:13:24.161700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a07d5e-da4a-4410-be97-63cb44ce06e5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:24.162233 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a07d5e-da4a-4410-be97-63cb44ce06e5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:24.163269 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a07d5e-da4a-4410-be97-63cb44ce06e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:24.186562 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a1a07d5e-da4a-4410-be97-63cb44ce06e5 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:24.188450 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a1a07d5e-da4a-4410-be97-63cb44ce06e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:24.197100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b7070-7e6a-49fd-8335-d89692bbc6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:24.197786 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b7070-7e6a-49fd-8335-d89692bbc6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:24.198107 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b7070-7e6a-49fd-8335-d89692bbc6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:24.198347 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b7070-7e6a-49fd-8335-d89692bbc6e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:24.199092 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b7070-7e6a-49fd-8335-d89692bbc6e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:24.199432 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b7070-7e6a-49fd-8335-d89692bbc6e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:24.201029 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b7070-7e6a-49fd-8335-d89692bbc6e4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:24.219228 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 214/420] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:13:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:24.226341 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b7070-7e6a-49fd-8335-d89692bbc6e4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:24.249055 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:24.296597 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 207/421] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:24.327893 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:24.348580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a07d5e-da4a-4410-be97-63cb44ce06e5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:24.424429 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b7070-7e6a-49fd-8335-d89692bbc6e4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:24.486015 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a07d5e-da4a-4410-be97-63cb44ce06e5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:24.507846 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1287bd1-88cb-456e-955c-ae02ef5eb511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:24.508732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1287bd1-88cb-456e-955c-ae02ef5eb511 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:24.509162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1287bd1-88cb-456e-955c-ae02ef5eb511 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:24.509546 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1287bd1-88cb-456e-955c-ae02ef5eb511 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:24.531054 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3433d-cc02-41a4-9e90-4cd107dd2f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:24.532844 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3433d-cc02-41a4-9e90-4cd107dd2f4d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:24.533990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3433d-cc02-41a4-9e90-4cd107dd2f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:24.534533 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3433d-cc02-41a4-9e90-4cd107dd2f4d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:24.559046 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc3433d-cc02-41a4-9e90-4cd107dd2f4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:24.567851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1fc3433d-cc02-41a4-9e90-4cd107dd2f4d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:13:24.567851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:24.567851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:24.567851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:24.567851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:24.567851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:24.567851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:24.567851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:24.567851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:24.567851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:24.567851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:24.567851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:24.567851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:24.567851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:24.567851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:24.567851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:24.569040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:24.569040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:24.569040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:24.569040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:24.569040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:24.569040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:24.569040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:24.569040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:24.569040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:24.569040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:24.569040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:24.569040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:24.569040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:24.569040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:24.569040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:24.569040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:24.570227 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:24.570227 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:24.570227 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:24.570227 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:24.570227 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:24.570227 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:24.570227 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:24.570227 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:24.570227 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:24.570227 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:24.570227 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 16:13:24.570227 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:24.587161 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 208/422] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:24.590629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3433d-cc02-41a4-9e90-4cd107dd2f4d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:24.642262 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1287bd1-88cb-456e-955c-ae02ef5eb511 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:24.643981 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1287bd1-88cb-456e-955c-ae02ef5eb511 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:24.646875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:24.717305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3433d-cc02-41a4-9e90-4cd107dd2f4d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:24.753656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a6ff9-743a-4f36-9a3e-0cadcc2c8507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:24.754717 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a6ff9-743a-4f36-9a3e-0cadcc2c8507 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:24.755208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a6ff9-743a-4f36-9a3e-0cadcc2c8507 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:24.755721 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a6ff9-743a-4f36-9a3e-0cadcc2c8507 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:24.757257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a6ff9-743a-4f36-9a3e-0cadcc2c8507 None admin] RBAC: Adding query filter params (name=member) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:24.758053 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a6ff9-743a-4f36-9a3e-0cadcc2c8507 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:24.759697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a6ff9-743a-4f36-9a3e-0cadcc2c8507 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:24.769834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 209/423] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:13:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:24.776026 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a6ff9-743a-4f36-9a3e-0cadcc2c8507 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:24.805142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:24.889955 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a6ff9-743a-4f36-9a3e-0cadcc2c8507 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:24.909273 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:24.927920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b3ef6-a1af-447e-89cf-47ad5958dbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:24.929490 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b3ef6-a1af-447e-89cf-47ad5958dbeb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:24.929979 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b3ef6-a1af-447e-89cf-47ad5958dbeb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:24.930557 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b3ef6-a1af-447e-89cf-47ad5958dbeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:24.939642 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 215/424] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 715 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 16:13:24.949667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1287bd1-88cb-456e-955c-ae02ef5eb511 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:25.044079 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b3ef6-a1af-447e-89cf-47ad5958dbeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:25.045948 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b3ef6-a1af-447e-89cf-47ad5958dbeb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:25.081361 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1287bd1-88cb-456e-955c-ae02ef5eb511 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:25.155049 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 210/425] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 16:13:25.155438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06d989-a89a-4119-a7d2-f71b225fd230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:25.155785 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:25.156540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06d989-a89a-4119-a7d2-f71b225fd230 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:25.156678 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06d989-a89a-4119-a7d2-f71b225fd230 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:25.156960 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06d989-a89a-4119-a7d2-f71b225fd230 None admin] PATH_INFO: `/v3/projects/e54d8744be3c4672b24fa2e19f024af0` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:25.158205 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06d989-a89a-4119-a7d2-f71b225fd230 None admin] RBAC: Authorizing `identity:get_project(project_id=e54d8744be3c4672b24fa2e19f024af0)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:25.160544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b3ef6-a1af-447e-89cf-47ad5958dbeb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:25.164271 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06d989-a89a-4119-a7d2-f71b225fd230 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:25.167008 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 216/426] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:13:24 2022] GET /identity/v3/projects/e54d8744be3c4672b24fa2e19f024af0 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:25.172496 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06d989-a89a-4119-a7d2-f71b225fd230 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:25.237843 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:25.267677 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b3ef6-a1af-447e-89cf-47ad5958dbeb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:25.309348 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfd796-f77b-4407-bb0a-40afa2ab6762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:25.310245 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfd796-f77b-4407-bb0a-40afa2ab6762 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:25.310533 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfd796-f77b-4407-bb0a-40afa2ab6762 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:25.310812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfd796-f77b-4407-bb0a-40afa2ab6762 None admin] PATH_INFO: `/v3/projects/195eb382a49146b6b27a5cdc500c1a05` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:25.312060 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bfd796-f77b-4407-bb0a-40afa2ab6762 None admin] RBAC: Authorizing `identity:get_project(project_id=195eb382a49146b6b27a5cdc500c1a05)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:25.314070 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bfd796-f77b-4407-bb0a-40afa2ab6762 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:25.316021 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 211/427] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:13:25 2022] GET /identity/v3/projects/195eb382a49146b6b27a5cdc500c1a05 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:25.321813 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfd796-f77b-4407-bb0a-40afa2ab6762 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:25.397747 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:25.432968 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfd796-f77b-4407-bb0a-40afa2ab6762 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:25.470736 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f933ab-fe42-4861-9c13-d0593e7fdf61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:25.471644 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f933ab-fe42-4861-9c13-d0593e7fdf61 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:25.471928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f933ab-fe42-4861-9c13-d0593e7fdf61 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:25.472203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f933ab-fe42-4861-9c13-d0593e7fdf61 None admin] PATH_INFO: `/v3/groups/026420b86aa24c14807e7f9fce4ef606` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:25.477256 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f933ab-fe42-4861-9c13-d0593e7fdf61 None admin] RBAC: Authorizing `identity:get_group(group_id=026420b86aa24c14807e7f9fce4ef606)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:25.478725 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f933ab-fe42-4861-9c13-d0593e7fdf61 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:25.487604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06d989-a89a-4119-a7d2-f71b225fd230 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:25.489273 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 212/428] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 30 16:13:25 2022] GET /identity/v3/groups/026420b86aa24c14807e7f9fce4ef606 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:25.491717 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f933ab-fe42-4861-9c13-d0593e7fdf61 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:25.548582 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0043c0a-3abe-4cf7-ba81-9b304a9cf1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:25.549400 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0043c0a-3abe-4cf7-ba81-9b304a9cf1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:25.549687 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0043c0a-3abe-4cf7-ba81-9b304a9cf1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:25.549966 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0043c0a-3abe-4cf7-ba81-9b304a9cf1f0 None admin] PATH_INFO: `/v3/groups/026420b86aa24c14807e7f9fce4ef606` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:25.551164 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0043c0a-3abe-4cf7-ba81-9b304a9cf1f0 None admin] RBAC: Authorizing `identity:get_group(group_id=026420b86aa24c14807e7f9fce4ef606)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:25.552658 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0043c0a-3abe-4cf7-ba81-9b304a9cf1f0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:25.554653 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 217/429] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 30 16:13:25 2022] GET /identity/v3/groups/026420b86aa24c14807e7f9fce4ef606 => generated 226 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:25.560764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5a4fe-aded-49c6-86ea-78d185de0925 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:25.561093 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5a4fe-aded-49c6-86ea-78d185de0925 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:25.561382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f5a4fe-aded-49c6-86ea-78d185de0925 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:25.563075 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 218/430] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:25.563875 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5a4fe-aded-49c6-86ea-78d185de0925 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:25.644422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f933ab-fe42-4861-9c13-d0593e7fdf61 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:25.682632 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e776175-b6c3-446c-8b8b-93b983c4a266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:25.683291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e776175-b6c3-446c-8b8b-93b983c4a266 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:25.683619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e776175-b6c3-446c-8b8b-93b983c4a266 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:25.683898 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e776175-b6c3-446c-8b8b-93b983c4a266 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:25.684730 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e776175-b6c3-446c-8b8b-93b983c4a266 None admin] RBAC: Adding query filter params (group.id=026420b86aa24c14807e7f9fce4ef606, role.id=9ab3e302ead54950b7c30103f1e7fda7, scope.project.id=195eb382a49146b6b27a5cdc500c1a05) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:25.685045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e776175-b6c3-446c-8b8b-93b983c4a266 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:25.686575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e776175-b6c3-446c-8b8b-93b983c4a266 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:25.688976 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:25.698353 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:25.699763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 213/431] 10.222.0.67 () {56 vars in 1376 bytes} [Thu Jun 30 16:13:25 2022] GET /identity/v3/role_assignments?group.id=026420b86aa24c14807e7f9fce4ef606&scope.project.id=195eb382a49146b6b27a5cdc500c1a05&role.id=9ab3e302ead54950b7c30103f1e7fda7 => generated 611 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:25.701269 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9a81a-8408-481f-9aa4-764576d93e8d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:25.701561 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9a81a-8408-481f-9aa4-764576d93e8d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:25.702017 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9a81a-8408-481f-9aa4-764576d93e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:25.710799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f5a4fe-aded-49c6-86ea-78d185de0925 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:25.745000 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-30e9a81a-8408-481f-9aa4-764576d93e8d None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:25.746399 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-30e9a81a-8408-481f-9aa4-764576d93e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:25.754451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7435d89-fdc4-458c-9dff-032919dba724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:25.755432 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7435d89-fdc4-458c-9dff-032919dba724 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:25.755902 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7435d89-fdc4-458c-9dff-032919dba724 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:25.758160 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7435d89-fdc4-458c-9dff-032919dba724 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:25.759270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7435d89-fdc4-458c-9dff-032919dba724 None admin] RBAC: Adding query filter params (group.id=026420b86aa24c14807e7f9fce4ef606, role.id=9ab3e302ead54950b7c30103f1e7fda7, scope.project.id=e54d8744be3c4672b24fa2e19f024af0) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:25.759768 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7435d89-fdc4-458c-9dff-032919dba724 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:25.761555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7435d89-fdc4-458c-9dff-032919dba724 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:25.770376 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 219/432] 10.222.0.67 () {56 vars in 1376 bytes} [Thu Jun 30 16:13:25 2022] GET /identity/v3/role_assignments?group.id=026420b86aa24c14807e7f9fce4ef606&scope.project.id=e54d8744be3c4672b24fa2e19f024af0&role.id=9ab3e302ead54950b7c30103f1e7fda7 => generated 611 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:25.851082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 214/433] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:25.878527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7435d89-fdc4-458c-9dff-032919dba724 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:26.011973 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7435d89-fdc4-458c-9dff-032919dba724 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:26.056871 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af78d34e-7184-4782-b693-c5be6cb42e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:26.057722 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78d34e-7184-4782-b693-c5be6cb42e02 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:26.058006 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78d34e-7184-4782-b693-c5be6cb42e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:26.058286 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af78d34e-7184-4782-b693-c5be6cb42e02 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:26.070947 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af78d34e-7184-4782-b693-c5be6cb42e02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:26.079433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-af78d34e-7184-4782-b693-c5be6cb42e02 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:13:26.079433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:26.079433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:26.079433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:26.079433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:26.079433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:26.079433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:26.079433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:26.079433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:26.079433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:26.079433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:26.079433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:26.079433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:26.079433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:26.079433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:26.079433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:26.080352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:26.080352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:26.080352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:26.080352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:26.080352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:26.080352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:26.080352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:26.080352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:26.080352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:26.080352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:26.080352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:26.080352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:26.080352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:26.080352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:26.080352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:26.080352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:26.081219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:26.081219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:26.081219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:26.081219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:26.081219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:26.081219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:26.081219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:26.081219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:26.081219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:26.081219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:26.081219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:13:26.081219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:26.081219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 220/434] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 30 16:13:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:26.087383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9a81a-8408-481f-9aa4-764576d93e8d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:26.107500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37244c-1ad0-47b6-866f-361d91f3ebeb None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:26.107774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37244c-1ad0-47b6-866f-361d91f3ebeb None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:26.107982 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37244c-1ad0-47b6-866f-361d91f3ebeb None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:26.110097 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 221/435] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:13:26.117666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5882b-ae04-4469-bc79-f1c2275c3714 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:26.117955 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5882b-ae04-4469-bc79-f1c2275c3714 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:26.118232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5882b-ae04-4469-bc79-f1c2275c3714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:26.138694 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-51d5882b-ae04-4469-bc79-f1c2275c3714 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:26.139975 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-51d5882b-ae04-4469-bc79-f1c2275c3714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:26.154714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:26.226579 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9a81a-8408-481f-9aa4-764576d93e8d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:26.238433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 222/436] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:13:26.258905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:26.259720 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d5882b-ae04-4469-bc79-f1c2275c3714 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:26.270969 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea8909-6bf1-4166-8f77-df9c0544c772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:26.270969 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea8909-6bf1-4166-8f77-df9c0544c772 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:26.270969 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea8909-6bf1-4166-8f77-df9c0544c772 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:26.270969 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea8909-6bf1-4166-8f77-df9c0544c772 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:26.280868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea8909-6bf1-4166-8f77-df9c0544c772 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:26.280868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea8909-6bf1-4166-8f77-df9c0544c772 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:26.280868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea8909-6bf1-4166-8f77-df9c0544c772 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:26.285011 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 215/437] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:13:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:26.290177 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea8909-6bf1-4166-8f77-df9c0544c772 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:26.381093 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d5882b-ae04-4469-bc79-f1c2275c3714 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:26.412212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea8909-6bf1-4166-8f77-df9c0544c772 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:26.419517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43292f4-2f30-4fb2-a7da-1d3aeb43985c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:26.420334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43292f4-2f30-4fb2-a7da-1d3aeb43985c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:26.420627 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43292f4-2f30-4fb2-a7da-1d3aeb43985c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:26.420903 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43292f4-2f30-4fb2-a7da-1d3aeb43985c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:26.431692 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43292f4-2f30-4fb2-a7da-1d3aeb43985c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:26.440120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-b43292f4-2f30-4fb2-a7da-1d3aeb43985c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:13:26.440120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:26.440120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:26.440120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:26.440120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:26.440120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:26.440120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:26.440120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:26.440120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:26.440120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:26.440120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:26.440120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:26.440120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:26.440120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:26.440120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:26.440120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:26.440995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:26.440995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:26.440995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:26.440995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:26.440995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:26.440995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:26.440995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:26.440995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:26.440995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:26.440995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:26.440995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:26.440995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:26.440995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:26.440995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:26.440995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:26.440995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:26.441781 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:26.441781 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:26.441781 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:26.441781 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:26.441781 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:26.441781 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:26.441781 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:26.441781 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:26.441781 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:26.441781 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:26.441781 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:13:26.441781 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:26.441781 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 223/438] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 30 16:13:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:13:26.447676 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43292f4-2f30-4fb2-a7da-1d3aeb43985c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:26.452396 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528102d-33a9-40cf-be60-232d6c590ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:26.453043 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528102d-33a9-40cf-be60-232d6c590ede None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:26.453320 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528102d-33a9-40cf-be60-232d6c590ede None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:26.453590 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528102d-33a9-40cf-be60-232d6c590ede None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:26.458098 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:26.515847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:26.528486 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f528102d-33a9-40cf-be60-232d6c590ede None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:26.529948 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f528102d-33a9-40cf-be60-232d6c590ede None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:26.557442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43292f4-2f30-4fb2-a7da-1d3aeb43985c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:26.597732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc728751-cad4-44cc-bade-cb8e625ad302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:26.598423 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc728751-cad4-44cc-bade-cb8e625ad302 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:26.598707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc728751-cad4-44cc-bade-cb8e625ad302 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:26.598985 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc728751-cad4-44cc-bade-cb8e625ad302 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:26.599780 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc728751-cad4-44cc-bade-cb8e625ad302 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:26.600141 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc728751-cad4-44cc-bade-cb8e625ad302 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:26.601679 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc728751-cad4-44cc-bade-cb8e625ad302 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:26.611274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 224/439] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:13:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:26.616921 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc728751-cad4-44cc-bade-cb8e625ad302 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:26.636731 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 216/440] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:13:26.641769 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528102d-33a9-40cf-be60-232d6c590ede None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:26.702760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:26.740134 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc728751-cad4-44cc-bade-cb8e625ad302 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:26.765053 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:26.775912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528102d-33a9-40cf-be60-232d6c590ede None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:26.795672 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd6bb9-1021-46c6-b317-46a0467d1e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:26.795672 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd6bb9-1021-46c6-b317-46a0467d1e70 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:26.795672 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd6bb9-1021-46c6-b317-46a0467d1e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:26.795672 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd6bb9-1021-46c6-b317-46a0467d1e70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:26.811880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adefa1c8-bc6b-43b0-a66d-fd7149942ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:26.812773 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefa1c8-bc6b-43b0-a66d-fd7149942ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:26.813060 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefa1c8-bc6b-43b0-a66d-fd7149942ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:26.813331 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adefa1c8-bc6b-43b0-a66d-fd7149942ea3 None admin] PATH_INFO: `/v3/projects/e54d8744be3c4672b24fa2e19f024af0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:26.814537 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adefa1c8-bc6b-43b0-a66d-fd7149942ea3 None admin] RBAC: Authorizing `identity:get_project(project_id=e54d8744be3c4672b24fa2e19f024af0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:26.816108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adefa1c8-bc6b-43b0-a66d-fd7149942ea3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:26.817753 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 217/441] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:13:26 2022] GET /identity/v3/projects/e54d8744be3c4672b24fa2e19f024af0 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:26.819614 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606b4ae-9d6a-4755-9c00-094b410ec175 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:26.819902 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606b4ae-9d6a-4755-9c00-094b410ec175 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:26.820362 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606b4ae-9d6a-4755-9c00-094b410ec175 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:26.821402 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 218/442] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:26.823856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b606b4ae-9d6a-4755-9c00-094b410ec175 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:26.853461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd6bb9-1021-46c6-b317-46a0467d1e70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:26.854946 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd6bb9-1021-46c6-b317-46a0467d1e70 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:26.925593 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:26.942732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b606b4ae-9d6a-4755-9c00-094b410ec175 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:26.962831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 225/443] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:13:26.964321 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc8a04-acc5-411f-843a-b486c77a31d2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:26.964612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc8a04-acc5-411f-843a-b486c77a31d2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:26.964889 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc8a04-acc5-411f-843a-b486c77a31d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:26.979898 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081469d7-62bc-4db2-b0d3-b572ed245f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:26.980477 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a9fc8a04-acc5-411f-843a-b486c77a31d2 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:26.980757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081469d7-62bc-4db2-b0d3-b572ed245f04 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:26.981032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081469d7-62bc-4db2-b0d3-b572ed245f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:26.981302 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081469d7-62bc-4db2-b0d3-b572ed245f04 None admin] PATH_INFO: `/v3/groups/026420b86aa24c14807e7f9fce4ef606` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:26.981669 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a9fc8a04-acc5-411f-843a-b486c77a31d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:26.982459 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081469d7-62bc-4db2-b0d3-b572ed245f04 None admin] RBAC: Authorizing `identity:get_group(group_id=026420b86aa24c14807e7f9fce4ef606)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:26.984018 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081469d7-62bc-4db2-b0d3-b572ed245f04 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:26.985884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 219/444] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 30 16:13:26 2022] GET /identity/v3/groups/026420b86aa24c14807e7f9fce4ef606 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:26.986528 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081469d7-62bc-4db2-b0d3-b572ed245f04 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:27.015856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:27.057715 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 226/445] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:27.058635 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc8a04-acc5-411f-843a-b486c77a31d2 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:27.089645 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081469d7-62bc-4db2-b0d3-b572ed245f04 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:27.124512 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb22a5-b8f4-4d16-8795-7b72577798f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:27.125380 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb22a5-b8f4-4d16-8795-7b72577798f4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:27.125660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb22a5-b8f4-4d16-8795-7b72577798f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:27.125945 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb22a5-b8f4-4d16-8795-7b72577798f4 None admin] PATH_INFO: `/v3/projects/195eb382a49146b6b27a5cdc500c1a05` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:27.127137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb22a5-b8f4-4d16-8795-7b72577798f4 None admin] RBAC: Authorizing `identity:get_project(project_id=195eb382a49146b6b27a5cdc500c1a05)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:27.128619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb22a5-b8f4-4d16-8795-7b72577798f4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:27.130297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 220/446] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:13:26 2022] GET /identity/v3/projects/195eb382a49146b6b27a5cdc500c1a05 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:27.131052 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb22a5-b8f4-4d16-8795-7b72577798f4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:27.181456 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:27.185268 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc8a04-acc5-411f-843a-b486c77a31d2 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:27.218226 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7391a5e-bf17-4bb9-a825-4e49376f3903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:27.218879 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7391a5e-bf17-4bb9-a825-4e49376f3903 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:27.219160 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7391a5e-bf17-4bb9-a825-4e49376f3903 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:27.219437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7391a5e-bf17-4bb9-a825-4e49376f3903 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:27.220342 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7391a5e-bf17-4bb9-a825-4e49376f3903 None admin] RBAC: Adding query filter params (group.id=026420b86aa24c14807e7f9fce4ef606, role.id=cfeec012f4ec48138c0e637e9a7fde61, scope.project.id=e54d8744be3c4672b24fa2e19f024af0) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:27.220691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7391a5e-bf17-4bb9-a825-4e49376f3903 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:27.222149 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7391a5e-bf17-4bb9-a825-4e49376f3903 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:27.226115 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:27.239991 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 227/447] 10.222.0.67 () {56 vars in 1376 bytes} [Thu Jun 30 16:13:27 2022] GET /identity/v3/role_assignments?group.id=026420b86aa24c14807e7f9fce4ef606&scope.project.id=e54d8744be3c4672b24fa2e19f024af0&role.id=cfeec012f4ec48138c0e637e9a7fde61 => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:27.239991 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7391a5e-bf17-4bb9-a825-4e49376f3903 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:27.240259 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb22a5-b8f4-4d16-8795-7b72577798f4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:27.274376 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac80d2d-18e1-49e0-9b3f-206bd7bd62fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:27.275638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac80d2d-18e1-49e0-9b3f-206bd7bd62fc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:27.276298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac80d2d-18e1-49e0-9b3f-206bd7bd62fc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:27.276583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac80d2d-18e1-49e0-9b3f-206bd7bd62fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:27.290887 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac80d2d-18e1-49e0-9b3f-206bd7bd62fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:27.298881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-eac80d2d-18e1-49e0-9b3f-206bd7bd62fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:13:27.298881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:27.298881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:27.298881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:27.298881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:27.298881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:27.298881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:27.298881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:27.298881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:27.298881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:27.298881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:27.298881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:27.298881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:27.298881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:27.298881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:27.298881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:27.300108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:27.300108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:27.300108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:27.300108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:27.300108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:27.300108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:27.300108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:27.300108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:27.300108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:27.300108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:27.300108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:27.300108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:27.300108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:27.300108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:27.300108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:27.300108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:27.300992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:27.300992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:27.300992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:27.300992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:27.300992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:27.300992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:27.300992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:27.300992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:27.300992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:27.300992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:27.300992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:13:27.300992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:27.300992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 221/448] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:13:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:27.307308 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac80d2d-18e1-49e0-9b3f-206bd7bd62fc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:27.366555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7391a5e-bf17-4bb9-a825-4e49376f3903 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:27.405551 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2fc11-dc19-46a3-a58a-184b4357aac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:27.406534 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2fc11-dc19-46a3-a58a-184b4357aac6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:27.407499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2fc11-dc19-46a3-a58a-184b4357aac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:27.407499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2fc11-dc19-46a3-a58a-184b4357aac6 None admin] PATH_INFO: `/v3/groups/026420b86aa24c14807e7f9fce4ef606` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:27.410530 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2fc11-dc19-46a3-a58a-184b4357aac6 None admin] RBAC: Authorizing `identity:get_group(group_id=026420b86aa24c14807e7f9fce4ef606)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:27.412065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2fc11-dc19-46a3-a58a-184b4357aac6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:27.414319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 228/449] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 30 16:13:27 2022] GET /identity/v3/groups/026420b86aa24c14807e7f9fce4ef606 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:27.420163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2fc11-dc19-46a3-a58a-184b4357aac6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:27.426150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac80d2d-18e1-49e0-9b3f-206bd7bd62fc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:27.454420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:27.476694 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5f164-101a-438d-b425-c7bc015850ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:27.477366 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5f164-101a-438d-b425-c7bc015850ce None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:27.477649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5f164-101a-438d-b425-c7bc015850ce None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:27.477968 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5f164-101a-438d-b425-c7bc015850ce None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:27.478785 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5f164-101a-438d-b425-c7bc015850ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:27.479091 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5f164-101a-438d-b425-c7bc015850ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:27.481680 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5f164-101a-438d-b425-c7bc015850ce None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:27.499464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 222/450] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:27.505040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5f164-101a-438d-b425-c7bc015850ce None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:27.527309 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:27.531986 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2fc11-dc19-46a3-a58a-184b4357aac6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:27.576695 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d017e-c7ee-4af9-b6cf-ca3d463d6a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:27.577369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d017e-c7ee-4af9-b6cf-ca3d463d6a4c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:27.577643 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d017e-c7ee-4af9-b6cf-ca3d463d6a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:27.577918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d017e-c7ee-4af9-b6cf-ca3d463d6a4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:27.578755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d017e-c7ee-4af9-b6cf-ca3d463d6a4c None admin] RBAC: Adding query filter params (group.id=026420b86aa24c14807e7f9fce4ef606, role.id=cfeec012f4ec48138c0e637e9a7fde61, scope.project.id=195eb382a49146b6b27a5cdc500c1a05) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:27.579069 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d017e-c7ee-4af9-b6cf-ca3d463d6a4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:27.580602 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d017e-c7ee-4af9-b6cf-ca3d463d6a4c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:27.588157 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 229/451] 10.222.0.67 () {56 vars in 1376 bytes} [Thu Jun 30 16:13:27 2022] GET /identity/v3/role_assignments?group.id=026420b86aa24c14807e7f9fce4ef606&scope.project.id=195eb382a49146b6b27a5cdc500c1a05&role.id=cfeec012f4ec48138c0e637e9a7fde61 => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:27.631808 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5f164-101a-438d-b425-c7bc015850ce None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:27.673447 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464f9f4-1e7b-4307-ac96-94a9b2c93665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:27.674121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464f9f4-1e7b-4307-ac96-94a9b2c93665 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:27.674413 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464f9f4-1e7b-4307-ac96-94a9b2c93665 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:27.674695 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464f9f4-1e7b-4307-ac96-94a9b2c93665 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:27.755443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9464f9f4-1e7b-4307-ac96-94a9b2c93665 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:27.757696 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9464f9f4-1e7b-4307-ac96-94a9b2c93665 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:27.777574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:27.862527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 223/452] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 16:13:27.868343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d017e-c7ee-4af9-b6cf-ca3d463d6a4c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:27.970018 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:27.977126 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d017e-c7ee-4af9-b6cf-ca3d463d6a4c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:28.011389 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403ae65-33ae-4525-be95-d89785defed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:28.012299 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403ae65-33ae-4525-be95-d89785defed4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:28.012781 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403ae65-33ae-4525-be95-d89785defed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:28.013272 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403ae65-33ae-4525-be95-d89785defed4 None admin] PATH_INFO: `/v3/projects/3967c884c6b647c2a8e60d73c62e7927` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:28.014833 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a403ae65-33ae-4525-be95-d89785defed4 None admin] RBAC: Authorizing `identity:get_project(project_id=3967c884c6b647c2a8e60d73c62e7927)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:28.016326 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a403ae65-33ae-4525-be95-d89785defed4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:28.018032 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 230/453] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:13:27 2022] GET /identity/v3/projects/3967c884c6b647c2a8e60d73c62e7927 => generated 344 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:28.024168 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464f9f4-1e7b-4307-ac96-94a9b2c93665 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:28.123415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464f9f4-1e7b-4307-ac96-94a9b2c93665 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:28.139730 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:28.157700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d14d4-9329-41c9-9a7d-80516b7a49f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:28.158536 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d14d4-9329-41c9-9a7d-80516b7a49f9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:28.158812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d14d4-9329-41c9-9a7d-80516b7a49f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:28.159079 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d14d4-9329-41c9-9a7d-80516b7a49f9 None admin] PATH_INFO: `/v3/groups/79ce9e920d8b4f3cbbba85006e4c647b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:28.160252 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d14d4-9329-41c9-9a7d-80516b7a49f9 None admin] RBAC: Authorizing `identity:get_group(group_id=79ce9e920d8b4f3cbbba85006e4c647b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:28.161729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d14d4-9329-41c9-9a7d-80516b7a49f9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:28.163895 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 224/454] 10.222.0.67 () {56 vars in 1180 bytes} [Thu Jun 30 16:13:28 2022] GET /identity/v3/groups/79ce9e920d8b4f3cbbba85006e4c647b => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:28.169913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403ae65-33ae-4525-be95-d89785defed4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:28.271252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403ae65-33ae-4525-be95-d89785defed4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:28.305653 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af6f58-d957-45a0-946f-9926d8f30107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:28.306298 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af6f58-d957-45a0-946f-9926d8f30107 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:28.306583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af6f58-d957-45a0-946f-9926d8f30107 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:28.306862 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af6f58-d957-45a0-946f-9926d8f30107 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:28.307735 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af6f58-d957-45a0-946f-9926d8f30107 None admin] RBAC: Adding query filter params (group.id=79ce9e920d8b4f3cbbba85006e4c647b, role.id=8acc34a163874a60b03728b814eaf075, scope.project.id=3967c884c6b647c2a8e60d73c62e7927) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:28.308071 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af6f58-d957-45a0-946f-9926d8f30107 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:28.309536 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af6f58-d957-45a0-946f-9926d8f30107 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:28.315388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 231/455] 10.222.0.67 () {56 vars in 1376 bytes} [Thu Jun 30 16:13:28 2022] GET /identity/v3/role_assignments?group.id=79ce9e920d8b4f3cbbba85006e4c647b&scope.project.id=3967c884c6b647c2a8e60d73c62e7927&role.id=8acc34a163874a60b03728b814eaf075 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:30.737822 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863352d9-224d-4ee3-9bf4-d594b6585421 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:30.738339 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863352d9-224d-4ee3-9bf4-d594b6585421 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:30.738414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863352d9-224d-4ee3-9bf4-d594b6585421 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:30.740069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 225/456] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:30.745064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb6477-9ac0-4378-89ed-2329950fffd8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:30.745517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb6477-9ac0-4378-89ed-2329950fffd8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:30.745626 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb6477-9ac0-4378-89ed-2329950fffd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:30.772525 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-4cfb6477-9ac0-4378-89ed-2329950fffd8 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:30.775123 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4cfb6477-9ac0-4378-89ed-2329950fffd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:30.803380 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:30.867119 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 232/457] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:13:30.918129 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863352d9-224d-4ee3-9bf4-d594b6585421 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:30.995664 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe115cd-5b43-4104-b2e6-4cd7704aef53 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:30.995664 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe115cd-5b43-4104-b2e6-4cd7704aef53 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:30.995664 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe115cd-5b43-4104-b2e6-4cd7704aef53 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:30.995664 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 233/458] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:31.001065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4da5cb-e6ee-4ed0-bcfb-d7ccbdb2db92 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:31.001354 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4da5cb-e6ee-4ed0-bcfb-d7ccbdb2db92 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:31.001630 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4da5cb-e6ee-4ed0-bcfb-d7ccbdb2db92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:31.053063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0d4da5cb-e6ee-4ed0-bcfb-d7ccbdb2db92 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:31.054316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0d4da5cb-e6ee-4ed0-bcfb-d7ccbdb2db92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:31.057563 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863352d9-224d-4ee3-9bf4-d594b6585421 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:31.116021 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7883b-3a24-4a6e-a419-2d2efcd9f6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:31.116837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7883b-3a24-4a6e-a419-2d2efcd9f6af None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:31.117114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7883b-3a24-4a6e-a419-2d2efcd9f6af None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:31.117385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7883b-3a24-4a6e-a419-2d2efcd9f6af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:31.130090 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab7883b-3a24-4a6e-a419-2d2efcd9f6af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:31.131816 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab7883b-3a24-4a6e-a419-2d2efcd9f6af None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:31.151279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-cab7883b-3a24-4a6e-a419-2d2efcd9f6af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:31.151279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:31.151279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:31.151279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:31.151279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:31.151279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:31.151279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:31.151279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:31.151279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:31.151279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:31.151279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:31.151279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:31.151279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:31.151279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:31.151279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:31.152422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:31.152422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:31.152422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:31.152422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:31.152422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:31.152422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:31.152422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:31.152422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:31.152422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:31.152422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:31.152422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:31.152422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:31.152422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:31.152422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:31.152422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:31.152422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:31.152422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:31.153307 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:31.153307 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:31.153307 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:31.153307 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:31.155444 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 226/459] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:31.156935 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625b418-e3c7-4a30-b2e3-74bd38367611 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:31.157396 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625b418-e3c7-4a30-b2e3-74bd38367611 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:31.157848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5625b418-e3c7-4a30-b2e3-74bd38367611 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:31.167494 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 234/461] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:13:31.167653 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 227/461] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:31.167728 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4da5cb-e6ee-4ed0-bcfb-d7ccbdb2db92 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:31.185152 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e347836-5950-476e-957b-7e0c023685eb None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:31.185439 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e347836-5950-476e-957b-7e0c023685eb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:31.185700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e347836-5950-476e-957b-7e0c023685eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:31.209708 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3e347836-5950-476e-957b-7e0c023685eb None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:31.211886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3e347836-5950-476e-957b-7e0c023685eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:31.250646 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:31.273713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:31.290884 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4da5cb-e6ee-4ed0-bcfb-d7ccbdb2db92 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:31.302077 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 228/462] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:13:31.303395 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e347836-5950-476e-957b-7e0c023685eb None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:31.328914 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba938ec-aac7-40c8-baf5-f846947d9af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:31.329746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba938ec-aac7-40c8-baf5-f846947d9af3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:31.330131 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba938ec-aac7-40c8-baf5-f846947d9af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:31.330534 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba938ec-aac7-40c8-baf5-f846947d9af3 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:31.331399 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba938ec-aac7-40c8-baf5-f846947d9af3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:31.331868 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba938ec-aac7-40c8-baf5-f846947d9af3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:31.333377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba938ec-aac7-40c8-baf5-f846947d9af3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:31.346406 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 235/463] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:31.348485 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba938ec-aac7-40c8-baf5-f846947d9af3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:31.432231 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e347836-5950-476e-957b-7e0c023685eb None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:31.469723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e1542c-2819-4f80-9fd0-ef8d2eb1e56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:31.470672 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1542c-2819-4f80-9fd0-ef8d2eb1e56b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:31.471076 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1542c-2819-4f80-9fd0-ef8d2eb1e56b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:31.471457 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1542c-2819-4f80-9fd0-ef8d2eb1e56b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:31.482453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e1542c-2819-4f80-9fd0-ef8d2eb1e56b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:31.484155 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e1542c-2819-4f80-9fd0-ef8d2eb1e56b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:31.487726 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba938ec-aac7-40c8-baf5-f846947d9af3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:31.503992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-17e1542c-2819-4f80-9fd0-ef8d2eb1e56b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:31.503992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:31.503992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:31.503992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:31.503992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:31.503992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:31.503992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:31.503992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:31.503992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:31.503992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:31.503992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:31.503992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:31.503992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:31.503992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:31.503992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:31.504905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:31.504905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:31.504905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:31.504905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:31.504905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:31.504905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:31.504905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:31.504905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:31.504905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:31.504905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:31.504905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:31.504905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:31.504905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:31.504905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:31.504905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:31.504905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:31.504905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:31.505740 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:31.505740 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:31.505740 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:31.505740 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:31.505948 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 229/464] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:31.511257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e1542c-2819-4f80-9fd0-ef8d2eb1e56b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:31.527828 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e0cdc-cb51-4584-8218-07656ef634a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:31.528494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:31.529694 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e0cdc-cb51-4584-8218-07656ef634a0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:31.529977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e0cdc-cb51-4584-8218-07656ef634a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:31.530247 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e0cdc-cb51-4584-8218-07656ef634a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:31.545563 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e0cdc-cb51-4584-8218-07656ef634a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:31.547060 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e0cdc-cb51-4584-8218-07656ef634a0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:31.557421 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:31.569431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-a14e0cdc-cb51-4584-8218-07656ef634a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:31.569431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:31.569431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:31.569431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:31.569431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:31.569431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:31.569431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:31.569431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:31.569431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:31.569431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:31.569431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:31.569431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:31.569431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:31.569431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:31.569431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:31.570420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:31.570420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:31.570420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:31.570420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:31.570420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:31.570420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:31.570420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:31.570420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:31.570420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:31.570420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:31.570420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:31.570420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:31.570420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:31.570420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:31.570420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:31.570420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:31.570420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:31.571282 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:31.571282 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:31.571282 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:31.571282 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:31.571282 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 236/465] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:31.572105 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e0cdc-cb51-4584-8218-07656ef634a0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:31.638741 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e1542c-2819-4f80-9fd0-ef8d2eb1e56b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:31.678090 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d69cce-fa46-4622-93f5-3ada739e1db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:31.678792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d69cce-fa46-4622-93f5-3ada739e1db1 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:31.679128 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d69cce-fa46-4622-93f5-3ada739e1db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:31.679917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d69cce-fa46-4622-93f5-3ada739e1db1 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:31.680945 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d69cce-fa46-4622-93f5-3ada739e1db1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:31.682487 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d69cce-fa46-4622-93f5-3ada739e1db1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:31.691661 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e0cdc-cb51-4584-8218-07656ef634a0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:31.728358 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951756e-e4f4-4933-b1b2-b6c5b91d659d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:31.729027 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951756e-e4f4-4933-b1b2-b6c5b91d659d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:31.729316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951756e-e4f4-4933-b1b2-b6c5b91d659d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:31.729597 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951756e-e4f4-4933-b1b2-b6c5b91d659d None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:31.730324 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951756e-e4f4-4933-b1b2-b6c5b91d659d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:31.730639 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951756e-e4f4-4933-b1b2-b6c5b91d659d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:31.732124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d951756e-e4f4-4933-b1b2-b6c5b91d659d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:31.749855 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 237/466] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:31.749855 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951756e-e4f4-4933-b1b2-b6c5b91d659d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:31.773260 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 230/467] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 30 16:13:31 2022] POST /identity/v3/users => generated 246 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:13:31.774908 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d69cce-fa46-4622-93f5-3ada739e1db1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:31.836362 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:31.868758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:31.898213 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951756e-e4f4-4933-b1b2-b6c5b91d659d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:31.900026 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d69cce-fa46-4622-93f5-3ada739e1db1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:31.946316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c328093-5f77-4834-b216-1f54ef02b459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:31.947223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c328093-5f77-4834-b216-1f54ef02b459 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:31.947757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c328093-5f77-4834-b216-1f54ef02b459 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:31.948175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c328093-5f77-4834-b216-1f54ef02b459 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:31.949117 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c328093-5f77-4834-b216-1f54ef02b459 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:31.949584 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c328093-5f77-4834-b216-1f54ef02b459 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:31.949938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2de216-007d-4b72-ba19-f31a1bbcc2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:31.950614 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2de216-007d-4b72-ba19-f31a1bbcc2ec None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:31.950895 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2de216-007d-4b72-ba19-f31a1bbcc2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:31.951169 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2de216-007d-4b72-ba19-f31a1bbcc2ec None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:31.951812 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c328093-5f77-4834-b216-1f54ef02b459 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:31.952023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2de216-007d-4b72-ba19-f31a1bbcc2ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:31.953456 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2de216-007d-4b72-ba19-f31a1bbcc2ec None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:31.967905 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 238/468] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:31.969289 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea5169-4e4c-4567-bccc-88769f68bced None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:31.969599 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea5169-4e4c-4567-bccc-88769f68bced None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:31.969873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea5169-4e4c-4567-bccc-88769f68bced None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:31.971233 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 239/469] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:31.975936 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea5169-4e4c-4567-bccc-88769f68bced None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:32.088555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea5169-4e4c-4567-bccc-88769f68bced None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:32.110236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:32.113318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 231/470] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 30 16:13:31 2022] POST /identity/v3/users => generated 248 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:13:32.121745 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93823581-46fe-46da-8da7-5cb974866413 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:32.122358 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93823581-46fe-46da-8da7-5cb974866413 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:32.122838 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93823581-46fe-46da-8da7-5cb974866413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:32.150276 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db81013f-4686-473c-ab55-68d4bd4d234e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:32.152110 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81013f-4686-473c-ab55-68d4bd4d234e None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:32.152453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81013f-4686-473c-ab55-68d4bd4d234e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:32.152727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db81013f-4686-473c-ab55-68d4bd4d234e None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:32.155261 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81013f-4686-473c-ab55-68d4bd4d234e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:32.159216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db81013f-4686-473c-ab55-68d4bd4d234e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:32.161173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-93823581-46fe-46da-8da7-5cb974866413 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:32.162906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-93823581-46fe-46da-8da7-5cb974866413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:32.254809 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 232/471] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:32.284652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93823581-46fe-46da-8da7-5cb974866413 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:32.291247 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 240/472] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 30 16:13:31 2022] POST /identity/v3/users => generated 248 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:13:32.342076 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ce18d-ac48-487e-bb0e-0808d444a6f4 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:32.342275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ce18d-ac48-487e-bb0e-0808d444a6f4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:32.342552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ce18d-ac48-487e-bb0e-0808d444a6f4 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:32.344384 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 241/473] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:13:32.350795 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a76837-7bb0-40b5-a9e2-7ad1d695e627 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:32.351123 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a76837-7bb0-40b5-a9e2-7ad1d695e627 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:32.351896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a76837-7bb0-40b5-a9e2-7ad1d695e627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:32.372284 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f8a76837-7bb0-40b5-a9e2-7ad1d695e627 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:32.373648 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f8a76837-7bb0-40b5-a9e2-7ad1d695e627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:32.374704 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:32.434630 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93823581-46fe-46da-8da7-5cb974866413 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:32.475505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 242/474] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:13:32.477966 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8252c2-15c5-40e3-8a19-9d1544db37b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:32.479016 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8252c2-15c5-40e3-8a19-9d1544db37b5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:32.479577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8252c2-15c5-40e3-8a19-9d1544db37b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:32.480069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8252c2-15c5-40e3-8a19-9d1544db37b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:32.493072 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8252c2-15c5-40e3-8a19-9d1544db37b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:32.494812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8252c2-15c5-40e3-8a19-9d1544db37b5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:32.499291 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a76837-7bb0-40b5-a9e2-7ad1d695e627 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:32.514867 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ae8252c2-15c5-40e3-8a19-9d1544db37b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:32.514867 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:32.514867 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:32.514867 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:32.514867 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:32.514867 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:32.514867 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:32.514867 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:32.514867 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:32.514867 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:32.514867 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:32.514867 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:32.514867 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:32.514867 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:32.514867 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:32.515782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:32.515782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:32.515782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:32.515782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:32.515782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:32.515782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:32.515782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:32.515782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:32.515782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:32.515782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:32.515782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:32.515782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:32.515782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:32.515782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:32.515782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:32.515782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:32.515782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:32.516605 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:32.516605 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:32.516605 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:32.516605 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:32.516605 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 233/475] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:13:32.524093 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8252c2-15c5-40e3-8a19-9d1544db37b5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:32.693036 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a76837-7bb0-40b5-a9e2-7ad1d695e627 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:32.697737 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:32.706039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8252c2-15c5-40e3-8a19-9d1544db37b5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:32.727858 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:32.757949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765ae24-2c30-4127-ae37-c3c863248ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:32.759241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765ae24-2c30-4127-ae37-c3c863248ce7 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:32.759540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765ae24-2c30-4127-ae37-c3c863248ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:32.759818 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7765ae24-2c30-4127-ae37-c3c863248ce7 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:32.760575 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765ae24-2c30-4127-ae37-c3c863248ce7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:32.762105 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7765ae24-2c30-4127-ae37-c3c863248ce7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:32.771702 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7154bf-38fa-4b1e-8d68-2f199f5cb6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:32.772413 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7154bf-38fa-4b1e-8d68-2f199f5cb6fc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:32.772691 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7154bf-38fa-4b1e-8d68-2f199f5cb6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:32.774197 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7154bf-38fa-4b1e-8d68-2f199f5cb6fc None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:32.775254 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7154bf-38fa-4b1e-8d68-2f199f5cb6fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:32.775591 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7154bf-38fa-4b1e-8d68-2f199f5cb6fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:32.777020 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7154bf-38fa-4b1e-8d68-2f199f5cb6fc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:32.790078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 234/476] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:32.796850 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7154bf-38fa-4b1e-8d68-2f199f5cb6fc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:32.828121 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.sql.core [None req-7765ae24-2c30-4127-ae37-c3c863248ce7 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 16:13:32.828121 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 30 16:13:32.828121 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [parameters: {'id': '3e9678ef566e4a2c9351ef2e1ded8658', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 30 16:13:32.828121 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80298) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 16:13:32.861726 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-7765ae24-2c30-4127-ae37-c3c863248ce7 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 30 16:13:32.861726 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:32.861726 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 16:13:32.861726 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 16:13:32.861726 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 16:13:32.861726 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 16:13:32.861726 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 16:13:32.861726 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 16:13:32.861726 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 16:13:32.861726 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application conn.query(q) Jun 30 16:13:32.861726 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 16:13:32.861726 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 16:13:32.861726 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 16:13:32.861726 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application result.read() Jun 30 16:13:32.861726 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 16:13:32.862698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 16:13:32.862698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 16:13:32.862698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 16:13:32.862698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 16:13:32.862698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 16:13:32.862698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 16:13:32.862698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 16:13:32.862698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 16:13:32.862698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:32.862698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 16:13:32.862698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:32.862698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:32.862698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 16:13:32.862698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 16:13:32.862698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 30 16:13:32.862698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return ref.to_dict() Jun 30 16:13:32.862698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 16:13:32.862698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application next(self.gen) Jun 30 16:13:32.867316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 16:13:32.867316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application yield resource Jun 30 16:13:32.867316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 16:13:32.867316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application next(self.gen) Jun 30 16:13:32.867316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 16:13:32.867316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 16:13:32.867316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 16:13:32.867316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 16:13:32.867316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 16:13:32.867316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise self.value Jun 30 16:13:32.867316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 16:13:32.867316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 16:13:32.867316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 16:13:32.867316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application session.commit() Jun 30 16:13:32.867316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 16:13:32.867316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 16:13:32.867316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 16:13:32.868241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 16:13:32.868241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 16:13:32.868241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self.session.flush() Jun 30 16:13:32.868241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 16:13:32.868241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 16:13:32.868241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 16:13:32.868241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 16:13:32.868241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 16:13:32.868241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application compat.raise_( Jun 30 16:13:32.868241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 16:13:32.868241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception Jun 30 16:13:32.868241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 16:13:32.868241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 16:13:32.868241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 16:13:32.868241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 16:13:32.868241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 16:13:32.868241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 16:13:32.869204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 16:13:32.869204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 16:13:32.869204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 30 16:13:32.869204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 30 16:13:32.869204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 16:13:32.869204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 16:13:32.869204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 16:13:32.869204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 16:13:32.869204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 16:13:32.869204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 16:13:32.869204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 16:13:32.869204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 16:13:32.869204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 16:13:32.869204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 16:13:32.869204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 16:13:32.869204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception Jun 30 16:13:32.870022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 16:13:32.870022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 16:13:32.870022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 16:13:32.870022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 16:13:32.870022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 16:13:32.870022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 16:13:32.870022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 16:13:32.870022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application conn.query(q) Jun 30 16:13:32.870022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 16:13:32.870022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 16:13:32.870022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 16:13:32.870022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application result.read() Jun 30 16:13:32.870022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 16:13:32.870022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 16:13:32.870022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 16:13:32.870022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 16:13:32.870788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 16:13:32.870788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 16:13:32.870788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 16:13:32.870788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 16:13:32.870788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 16:13:32.870788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 30 16:13:32.870788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [parameters: {'id': '3e9678ef566e4a2c9351ef2e1ded8658', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 30 16:13:32.870788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 16:13:32.870788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:32.870788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:13:32.870788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:32.870788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:32.870788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:32.870788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:32.870788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:32.870788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:32.871589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:32.871589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:32.871589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:32.871589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:32.871589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:32.871589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:32.871589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 30 16:13:32.871589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 30 16:13:32.871589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:32.871589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:32.871589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 30 16:13:32.871589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 30 16:13:32.871589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 16:13:32.871589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 16:13:32.871589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 30 16:13:32.871589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:32.872433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 243/477] 10.222.0.67 () {60 vars in 1131 bytes} [Thu Jun 30 16:13:32 2022] POST /identity/v3/roles => generated 144 bytes in 365 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:13:32.884662 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765ae24-2c30-4127-ae37-c3c863248ce7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:32.921929 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7154bf-38fa-4b1e-8d68-2f199f5cb6fc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:32.965162 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ea463-940a-426e-81b9-fe1a98de19ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:32.965889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ea463-940a-426e-81b9-fe1a98de19ee None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:32.966179 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ea463-940a-426e-81b9-fe1a98de19ee None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:32.966495 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9ea463-940a-426e-81b9-fe1a98de19ee None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:32.967223 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ea463-940a-426e-81b9-fe1a98de19ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:32.969301 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9ea463-940a-426e-81b9-fe1a98de19ee None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:33.053707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7765ae24-2c30-4127-ae37-c3c863248ce7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:33.094330 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c1509-c9fd-4c3e-a8b4-4f16ff53ffbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:33.095453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c1509-c9fd-4c3e-a8b4-4f16ff53ffbe None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:33.095943 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c1509-c9fd-4c3e-a8b4-4f16ff53ffbe None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:33.096367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c1509-c9fd-4c3e-a8b4-4f16ff53ffbe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:33.107785 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c1509-c9fd-4c3e-a8b4-4f16ff53ffbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:33.122412 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 235/478] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 30 16:13:32 2022] POST /identity/v3/users => generated 249 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:13:33.123913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-c30c1509-c9fd-4c3e-a8b4-4f16ff53ffbe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:13:33.123913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:33.123913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:33.123913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:33.123913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:33.123913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:33.123913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:33.123913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:33.123913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:33.123913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:33.123913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:33.123913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:33.123913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:33.123913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:33.123913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:33.123913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:33.125007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:33.125007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:33.125007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:33.125007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:33.125007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:33.125007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:33.125007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:33.125007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:33.125007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:33.125007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:33.125007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:33.125007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:33.125007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:33.125007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:33.125007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:33.125007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:33.126017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:33.126017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:33.126017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:33.126017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:33.126017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:33.126017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:33.126017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:33.126017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:33.126017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:33.126017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:33.126017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 16:13:33.126017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:33.128569 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 244/479] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 30 16:13:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:33.136723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ea463-940a-426e-81b9-fe1a98de19ee None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:33.188893 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:33.293977 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9ea463-940a-426e-81b9-fe1a98de19ee None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:33.339871 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ecd9c7-1e2a-4932-8d9a-28999ba143e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:33.340548 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ecd9c7-1e2a-4932-8d9a-28999ba143e1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:33.340826 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ecd9c7-1e2a-4932-8d9a-28999ba143e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:33.341102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ecd9c7-1e2a-4932-8d9a-28999ba143e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:33.341853 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ecd9c7-1e2a-4932-8d9a-28999ba143e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:33.342158 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ecd9c7-1e2a-4932-8d9a-28999ba143e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:33.343555 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ecd9c7-1e2a-4932-8d9a-28999ba143e1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:33.352426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 236/480] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:13:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:34.209481 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93494725-34e4-44fd-a909-9ff4a48bd761 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:34.210545 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93494725-34e4-44fd-a909-9ff4a48bd761 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:34.211041 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93494725-34e4-44fd-a909-9ff4a48bd761 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:34.212697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 245/481] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:34.225024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3eedee-00e7-4fc9-966d-d150861e77b5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:34.226618 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3eedee-00e7-4fc9-966d-d150861e77b5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:34.227109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3eedee-00e7-4fc9-966d-d150861e77b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:34.251330 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-aa3eedee-00e7-4fc9-966d-d150861e77b5 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:34.252880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-aa3eedee-00e7-4fc9-966d-d150861e77b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:34.345385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 237/482] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:34.383714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93494725-34e4-44fd-a909-9ff4a48bd761 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:34.422685 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:34.517291 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93494725-34e4-44fd-a909-9ff4a48bd761 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:34.563211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468e048-9901-4cb1-bf8e-675ef254b2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:34.564426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468e048-9901-4cb1-bf8e-675ef254b2b7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:34.564912 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468e048-9901-4cb1-bf8e-675ef254b2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:34.565389 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468e048-9901-4cb1-bf8e-675ef254b2b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:34.589829 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468e048-9901-4cb1-bf8e-675ef254b2b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:34.598457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-d468e048-9901-4cb1-bf8e-675ef254b2b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:34.598457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:34.598457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:34.598457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:34.598457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:34.598457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:34.598457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:34.598457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:34.598457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:34.598457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:34.598457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:34.598457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:34.598457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:34.598457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:34.598457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:34.598457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:34.599422 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:34.599422 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:34.599422 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:34.599422 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:34.599422 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:34.599422 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:34.599422 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:34.599422 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:34.599422 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:34.599422 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:34.599422 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:34.599422 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:34.599422 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:34.599422 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:34.599422 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:34.599422 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:34.604585 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:34.604585 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:34.604585 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:34.604585 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:34.604585 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:34.604585 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:34.604585 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:34.604585 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:34.604585 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:34.604585 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:34.604585 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:34.604585 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:34.606844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 246/483] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 30 16:13:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:13:34.612638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3eedee-00e7-4fc9-966d-d150861e77b5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:34.639888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:34.754300 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3eedee-00e7-4fc9-966d-d150861e77b5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:34.803134 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7faa9a-acc1-46f6-bc6c-65a582cacc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:34.803817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7faa9a-acc1-46f6-bc6c-65a582cacc62 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:34.804638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7faa9a-acc1-46f6-bc6c-65a582cacc62 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:34.804934 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7faa9a-acc1-46f6-bc6c-65a582cacc62 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:34.805884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7faa9a-acc1-46f6-bc6c-65a582cacc62 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:34.806221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7faa9a-acc1-46f6-bc6c-65a582cacc62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:34.807671 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7faa9a-acc1-46f6-bc6c-65a582cacc62 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:34.816774 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 238/484] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 30 16:13:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:34.822470 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468e048-9901-4cb1-bf8e-675ef254b2b7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:34.948940 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abffcf4-d673-4ad3-97ce-a3303a2438cf None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:34.949599 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abffcf4-d673-4ad3-97ce-a3303a2438cf None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:34.950072 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abffcf4-d673-4ad3-97ce-a3303a2438cf None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:34.951937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 239/485] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:13:34.957300 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae2513-bdac-4bbd-8432-99304e262715 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:34.957936 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae2513-bdac-4bbd-8432-99304e262715 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:34.958451 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae2513-bdac-4bbd-8432-99304e262715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:34.967813 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:34.980017 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fdae2513-bdac-4bbd-8432-99304e262715 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:34.981476 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fdae2513-bdac-4bbd-8432-99304e262715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:34.988610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468e048-9901-4cb1-bf8e-675ef254b2b7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:35.053786 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7338b-3a3b-442e-a01d-612171304673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:35.055006 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7338b-3a3b-442e-a01d-612171304673 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:35.055288 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7338b-3a3b-442e-a01d-612171304673 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:35.055582 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7338b-3a3b-442e-a01d-612171304673 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:35.075750 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd7338b-3a3b-442e-a01d-612171304673 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:35.077223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd7338b-3a3b-442e-a01d-612171304673 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:35.117750 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 240/486] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:35.118697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-9cd7338b-3a3b-442e-a01d-612171304673 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:35.118697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:35.118697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:35.118697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:35.118697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:35.118697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:35.118697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:35.118697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:35.118697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:35.118697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:35.118697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:35.118697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:35.118697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:35.118697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:35.118697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:35.119623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:35.119623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:35.119623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:35.119623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:35.119623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:35.119623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:35.119623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:35.119623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:35.119623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:35.119623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:35.119623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:35.119623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:35.119623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:35.119623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:35.119623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:35.119623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:35.119623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:35.120830 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:35.120830 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:35.120830 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:35.120830 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:35.120830 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 247/487] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:35.125502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae2513-bdac-4bbd-8432-99304e262715 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:35.140774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7338b-3a3b-442e-a01d-612171304673 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:35.153226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:35.298286 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7338b-3a3b-442e-a01d-612171304673 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:35.301565 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae2513-bdac-4bbd-8432-99304e262715 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:35.338226 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22738d9f-ac67-4775-8e76-900a8139e1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:35.339047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b376035-a3b7-49b3-b87c-66e36f51136d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:35.339295 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22738d9f-ac67-4775-8e76-900a8139e1db None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:35.339608 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22738d9f-ac67-4775-8e76-900a8139e1db None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:35.339712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b376035-a3b7-49b3-b87c-66e36f51136d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:35.339919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22738d9f-ac67-4775-8e76-900a8139e1db None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:35.340019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b376035-a3b7-49b3-b87c-66e36f51136d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:35.340259 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b376035-a3b7-49b3-b87c-66e36f51136d None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:35.340979 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b376035-a3b7-49b3-b87c-66e36f51136d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:35.341279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b376035-a3b7-49b3-b87c-66e36f51136d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:35.342847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b376035-a3b7-49b3-b87c-66e36f51136d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:35.348791 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:35.352472 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22738d9f-ac67-4775-8e76-900a8139e1db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:35.356369 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 241/488] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:35.357795 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc99fa-3070-4d20-a3dd-86d5057e5547 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:35.358139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc99fa-3070-4d20-a3dd-86d5057e5547 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:35.358412 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc99fa-3070-4d20-a3dd-86d5057e5547 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:35.360041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 242/489] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:35.364743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-22738d9f-ac67-4775-8e76-900a8139e1db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:35.364743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:35.364743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:35.364743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:35.364743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:35.364743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:35.364743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:35.364743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:35.364743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:35.364743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:35.364743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:35.364743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:35.364743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:35.364743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:35.364743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:35.364743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:35.367275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:35.367275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:35.367275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:35.367275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:35.367275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:35.367275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:35.367275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:35.367275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:35.367275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:35.367275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:35.367275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:35.367275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:35.367275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:35.367275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:35.367275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:35.367275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:35.368779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:35.368779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:35.368779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:35.368779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:35.368779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:35.368779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:35.368779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:35.368779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:35.368779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:35.368779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:35.368779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:35.368779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:35.368779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 248/490] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 30 16:13:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:13:35.374975 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22738d9f-ac67-4775-8e76-900a8139e1db None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:35.375111 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bda2c4-6d5d-4c02-ac6e-9025d43b56bd None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:35.375111 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bda2c4-6d5d-4c02-ac6e-9025d43b56bd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:35.375111 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bda2c4-6d5d-4c02-ac6e-9025d43b56bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:35.392441 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e4bda2c4-6d5d-4c02-ac6e-9025d43b56bd None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:35.393742 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e4bda2c4-6d5d-4c02-ac6e-9025d43b56bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:35.469227 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 243/491] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:35.470093 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bda2c4-6d5d-4c02-ac6e-9025d43b56bd None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:35.478981 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:35.494219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22738d9f-ac67-4775-8e76-900a8139e1db None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:35.533033 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63707fba-173b-45d3-8efb-a6c8199e18a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:35.534070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63707fba-173b-45d3-8efb-a6c8199e18a6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:35.534493 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63707fba-173b-45d3-8efb-a6c8199e18a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:35.534910 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63707fba-173b-45d3-8efb-a6c8199e18a6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:35.545432 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63707fba-173b-45d3-8efb-a6c8199e18a6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:35.547215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63707fba-173b-45d3-8efb-a6c8199e18a6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:35.564917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-63707fba-173b-45d3-8efb-a6c8199e18a6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 16:13:35.564917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:35.564917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:35.564917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:35.564917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:35.564917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:35.564917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:35.564917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:35.564917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:35.564917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:35.564917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:35.564917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:35.564917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:13:35.564917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:13:35.564917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:13:35.564917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:13:35.566074 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:35.566074 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:35.566074 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:13:35.566074 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:35.566074 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:13:35.566074 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:35.566074 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:35.566074 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:35.566074 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:35.566074 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:35.566074 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:35.566074 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:35.566074 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:35.566074 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:35.566074 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:35.566074 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:35.566074 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:35.566074 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:35.567027 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:35.567027 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:35.567027 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:13:35.567027 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:13:35.567027 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:13:35.567027 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:13:35.567027 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:13:35.567027 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:13:35.567027 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 16:13:35.567027 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:35.569220 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 249/492] 10.222.0.67 () {56 vars in 1093 bytes} [Thu Jun 30 16:13:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:35.575529 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63707fba-173b-45d3-8efb-a6c8199e18a6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:35.599881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bda2c4-6d5d-4c02-ac6e-9025d43b56bd None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:35.635267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db3cdb-5a41-43b6-ab1b-fac6b235709c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:35.635971 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db3cdb-5a41-43b6-ab1b-fac6b235709c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:35.636256 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db3cdb-5a41-43b6-ab1b-fac6b235709c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:35.636532 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db3cdb-5a41-43b6-ab1b-fac6b235709c None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:35.637279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db3cdb-5a41-43b6-ab1b-fac6b235709c None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:35.637585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db3cdb-5a41-43b6-ab1b-fac6b235709c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:35.639042 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db3cdb-5a41-43b6-ab1b-fac6b235709c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:35.643421 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:35.649991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 244/493] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 30 16:13:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:35.651158 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac81fc-fe36-410e-bc4d-7b41583083fe None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:35.668894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac81fc-fe36-410e-bc4d-7b41583083fe None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:35.668894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac81fc-fe36-410e-bc4d-7b41583083fe None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:35.668894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 245/494] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:35.668894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac81fc-fe36-410e-bc4d-7b41583083fe None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:35.693862 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63707fba-173b-45d3-8efb-a6c8199e18a6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:35.720460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:35.731626 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150a5ae-4772-411a-942e-3099cac1b340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:35.732475 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150a5ae-4772-411a-942e-3099cac1b340 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:35.732754 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150a5ae-4772-411a-942e-3099cac1b340 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:35.733025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150a5ae-4772-411a-942e-3099cac1b340 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:35.743808 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150a5ae-4772-411a-942e-3099cac1b340 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:35.751865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-9150a5ae-4772-411a-942e-3099cac1b340 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:35.751865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:35.751865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:35.751865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:35.751865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:35.751865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:35.751865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:35.751865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:35.751865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:35.751865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:35.751865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:35.751865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:35.751865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:35.751865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:35.751865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:35.751865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:35.752796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:35.752796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:35.752796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:35.752796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:35.752796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:35.752796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:35.752796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:35.752796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:35.752796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:35.752796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:35.752796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:35.752796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:35.752796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:35.752796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:35.752796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:35.752796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:35.753596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:35.753596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:35.753596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:35.753596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:35.753596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:35.753596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:35.753596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:35.753596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:35.753596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:35.753596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:35.753596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:35.753596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:35.753596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 250/495] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 30 16:13:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:35.754239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150a5ae-4772-411a-942e-3099cac1b340 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:35.768214 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac81fc-fe36-410e-bc4d-7b41583083fe None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:35.806288 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b7473-ba57-41bb-b9a8-8da1f2c9684e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:35.807351 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b7473-ba57-41bb-b9a8-8da1f2c9684e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:35.807832 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b7473-ba57-41bb-b9a8-8da1f2c9684e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:35.808337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b7473-ba57-41bb-b9a8-8da1f2c9684e None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:35.809371 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204b7473-ba57-41bb-b9a8-8da1f2c9684e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:35.809865 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204b7473-ba57-41bb-b9a8-8da1f2c9684e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:35.811593 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204b7473-ba57-41bb-b9a8-8da1f2c9684e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:35.851873 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 246/496] 10.222.0.67 () {56 vars in 1133 bytes} [Thu Jun 30 16:13:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:35.853422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e43c2-a569-4a6b-99ee-3848d6ac5a72 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:35.853710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e43c2-a569-4a6b-99ee-3848d6ac5a72 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:35.853983 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100e43c2-a569-4a6b-99ee-3848d6ac5a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:35.863333 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150a5ae-4772-411a-942e-3099cac1b340 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:35.871138 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-100e43c2-a569-4a6b-99ee-3848d6ac5a72 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:35.872359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-100e43c2-a569-4a6b-99ee-3848d6ac5a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:35.886305 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:35.902696 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d156d8-c671-4b0f-ae37-03bda7385718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:35.909095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d156d8-c671-4b0f-ae37-03bda7385718 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:35.909095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d156d8-c671-4b0f-ae37-03bda7385718 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:35.909095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d156d8-c671-4b0f-ae37-03bda7385718 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:35.914828 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d156d8-c671-4b0f-ae37-03bda7385718 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:35.916330 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d156d8-c671-4b0f-ae37-03bda7385718 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:35.934166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-26d156d8-c671-4b0f-ae37-03bda7385718 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:35.934166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:35.934166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:35.934166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:35.934166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:35.934166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:35.934166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:35.934166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:35.934166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:35.934166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:35.934166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:35.934166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:35.934166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:35.934166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:35.934166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:35.935118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:35.935118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:35.935118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:35.935118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:35.935118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:35.935118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:35.935118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:35.935118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:35.935118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:35.935118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:35.935118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:35.935118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:35.935118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:35.935118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:35.935118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:35.935118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:35.935118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:35.935975 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:35.935975 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:35.935975 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:35.935975 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:35.935975 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 251/497] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:35.936244 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d156d8-c671-4b0f-ae37-03bda7385718 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:35.946930 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 247/498] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:35.947807 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e43c2-a569-4a6b-99ee-3848d6ac5a72 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:36.005727 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:36.042544 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d156d8-c671-4b0f-ae37-03bda7385718 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:36.061163 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100e43c2-a569-4a6b-99ee-3848d6ac5a72 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:36.077701 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e4569d-49bd-43e5-869f-eb0c20db086a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:36.078350 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4569d-49bd-43e5-869f-eb0c20db086a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:36.078634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4569d-49bd-43e5-869f-eb0c20db086a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:36.078917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4569d-49bd-43e5-869f-eb0c20db086a None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:36.079694 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e4569d-49bd-43e5-869f-eb0c20db086a None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:36.080026 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e4569d-49bd-43e5-869f-eb0c20db086a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:36.081449 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e4569d-49bd-43e5-869f-eb0c20db086a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:36.089902 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 252/499] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 30 16:13:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:36.090935 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732db769-0825-4ae2-9b94-27b193ef80c4 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:36.091227 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732db769-0825-4ae2-9b94-27b193ef80c4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:36.091566 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732db769-0825-4ae2-9b94-27b193ef80c4 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:36.092810 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 253/500] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:36.093511 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732db769-0825-4ae2-9b94-27b193ef80c4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:36.101570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca4f0d-c3a9-4a0d-b47b-d2c9e1cf7232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:36.102269 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca4f0d-c3a9-4a0d-b47b-d2c9e1cf7232 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:36.102495 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca4f0d-c3a9-4a0d-b47b-d2c9e1cf7232 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:36.102769 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca4f0d-c3a9-4a0d-b47b-d2c9e1cf7232 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:36.129988 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:36.172994 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aca4f0d-c3a9-4a0d-b47b-d2c9e1cf7232 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:36.174490 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aca4f0d-c3a9-4a0d-b47b-d2c9e1cf7232 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:36.199158 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732db769-0825-4ae2-9b94-27b193ef80c4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:36.227226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:36.233835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78e414-e6d6-47fa-b1e4-abfd3701458b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:36.234485 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78e414-e6d6-47fa-b1e4-abfd3701458b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:36.234778 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78e414-e6d6-47fa-b1e4-abfd3701458b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:36.235057 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78e414-e6d6-47fa-b1e4-abfd3701458b None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:36.235811 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78e414-e6d6-47fa-b1e4-abfd3701458b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:36.236123 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78e414-e6d6-47fa-b1e4-abfd3701458b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:36.237556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a78e414-e6d6-47fa-b1e4-abfd3701458b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:36.250307 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 254/501] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:36.250307 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78e414-e6d6-47fa-b1e4-abfd3701458b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:36.279308 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 248/502] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:13:36.280193 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca4f0d-c3a9-4a0d-b47b-d2c9e1cf7232 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:36.347129 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:36.372411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78e414-e6d6-47fa-b1e4-abfd3701458b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:36.382617 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca4f0d-c3a9-4a0d-b47b-d2c9e1cf7232 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:36.389240 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42ef3ea-9795-41d5-a790-b88c71edbe22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:36.390039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ef3ea-9795-41d5-a790-b88c71edbe22 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:36.390320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ef3ea-9795-41d5-a790-b88c71edbe22 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:36.390596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42ef3ea-9795-41d5-a790-b88c71edbe22 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:36.401814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42ef3ea-9795-41d5-a790-b88c71edbe22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:36.410550 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-a42ef3ea-9795-41d5-a790-b88c71edbe22 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:36.410550 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:36.410550 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:36.410550 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:36.410550 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:36.410550 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:36.410550 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:36.410550 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:36.410550 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:36.410550 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:36.410550 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:36.410550 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:36.410550 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:36.410550 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:36.410550 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:36.410550 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:36.411486 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:36.411486 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:36.411486 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:36.411486 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:36.411486 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:36.411486 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:36.411486 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:36.411486 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:36.411486 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:36.411486 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:36.411486 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:36.411486 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:36.411486 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:36.411486 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:36.411486 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:36.411486 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:36.412318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:36.412318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:36.412318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:36.412318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:36.412318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:36.412318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:36.412318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:36.412318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:36.412318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:36.412318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:36.412318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:36.412318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:36.412318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 255/503] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 30 16:13:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:36.413882 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafc0d8-3fbb-4b4a-a320-e8b64b0ae92a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:36.414044 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafc0d8-3fbb-4b4a-a320-e8b64b0ae92a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:36.414477 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafc0d8-3fbb-4b4a-a320-e8b64b0ae92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:36.420997 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df71198-75c8-4ed8-ad63-00f642986639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:36.421797 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df71198-75c8-4ed8-ad63-00f642986639 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:36.422079 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df71198-75c8-4ed8-ad63-00f642986639 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:36.422360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df71198-75c8-4ed8-ad63-00f642986639 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:36.433348 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df71198-75c8-4ed8-ad63-00f642986639 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:36.434363 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0dafc0d8-3fbb-4b4a-a320-e8b64b0ae92a None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:36.434835 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df71198-75c8-4ed8-ad63-00f642986639 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:36.435617 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0dafc0d8-3fbb-4b4a-a320-e8b64b0ae92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:36.454468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-8df71198-75c8-4ed8-ad63-00f642986639 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:36.454468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:36.454468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:36.454468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:36.454468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:36.454468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:36.454468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:36.454468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:36.454468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:36.454468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:36.454468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:36.454468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:36.454468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:36.454468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:36.454468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:36.455638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:36.455638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:36.455638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:36.455638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:36.455638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:36.455638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:36.455638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:36.455638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:36.455638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:36.455638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:36.455638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:36.455638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:36.455638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:36.455638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:36.455638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:36.455638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:36.455638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:36.456577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:36.456577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:36.456577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:36.456577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:36.456577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 249/504] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:36.456850 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df71198-75c8-4ed8-ad63-00f642986639 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:36.461303 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:36.508528 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 256/505] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:36.509243 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafc0d8-3fbb-4b4a-a320-e8b64b0ae92a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:36.567099 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df71198-75c8-4ed8-ad63-00f642986639 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:36.595236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:36.601226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab24d6-fb44-4b30-8af5-98d30309c3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:36.602176 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab24d6-fb44-4b30-8af5-98d30309c3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:36.602478 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab24d6-fb44-4b30-8af5-98d30309c3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:36.602736 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab24d6-fb44-4b30-8af5-98d30309c3e2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:36.610272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cab24d6-fb44-4b30-8af5-98d30309c3e2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:36.610503 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafc0d8-3fbb-4b4a-a320-e8b64b0ae92a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:36.611806 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cab24d6-fb44-4b30-8af5-98d30309c3e2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:36.627953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9cab24d6-fb44-4b30-8af5-98d30309c3e2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 16:13:36.627953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:36.627953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:36.627953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:36.627953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:36.627953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:36.627953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:36.627953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:36.627953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:36.627953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:36.627953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:36.627953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:36.627953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:13:36.627953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:13:36.627953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:13:36.627953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:13:36.628868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:36.628868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:36.628868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:13:36.628868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:36.628868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:13:36.628868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:36.628868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:36.628868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:36.628868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:36.628868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:36.628868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:36.628868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:36.628868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:36.628868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:36.628868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:36.628868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:36.628868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:36.628868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:36.629744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:36.629744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:36.629744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:13:36.629744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:13:36.629744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:13:36.629744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:13:36.629744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:13:36.629744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:13:36.629744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 16:13:36.629744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:36.631623 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 250/506] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:36.633855 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab24d6-fb44-4b30-8af5-98d30309c3e2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:36.645442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafddc02-5427-49be-bb6b-805790e1a2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:36.646263 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafddc02-5427-49be-bb6b-805790e1a2eb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:36.646549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafddc02-5427-49be-bb6b-805790e1a2eb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:36.646829 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafddc02-5427-49be-bb6b-805790e1a2eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:36.657466 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafddc02-5427-49be-bb6b-805790e1a2eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:36.658916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafddc02-5427-49be-bb6b-805790e1a2eb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:36.681070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-eafddc02-5427-49be-bb6b-805790e1a2eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:36.681070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:36.681070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:36.681070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:36.681070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:36.681070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:36.681070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:36.681070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:36.681070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:36.681070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:36.681070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:36.681070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:36.681070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:36.681070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:36.681070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:36.681980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:36.681980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:36.681980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:36.681980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:36.681980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:36.681980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:36.681980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:36.681980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:36.681980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:36.681980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:36.681980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:36.681980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:36.681980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:36.681980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:36.681980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:36.681980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:36.681980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:36.682823 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:36.682823 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:36.682823 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:36.682823 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:36.682823 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 257/507] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:36.683287 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafddc02-5427-49be-bb6b-805790e1a2eb None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:36.701179 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:36.739477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab24d6-fb44-4b30-8af5-98d30309c3e2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:36.775119 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44c8a5-439e-4862-931c-6222237d06c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:36.775816 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44c8a5-439e-4862-931c-6222237d06c2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:36.776102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44c8a5-439e-4862-931c-6222237d06c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:36.776383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44c8a5-439e-4862-931c-6222237d06c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:36.777123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44c8a5-439e-4862-931c-6222237d06c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:36.777433 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44c8a5-439e-4862-931c-6222237d06c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:36.778829 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44c8a5-439e-4862-931c-6222237d06c2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:36.786677 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafddc02-5427-49be-bb6b-805790e1a2eb None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:36.791566 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 251/508] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 30 16:13:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:36.791566 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44c8a5-439e-4862-931c-6222237d06c2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:36.821907 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f606d0-bedb-4061-9e0d-60bdd60e1010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:36.822653 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f606d0-bedb-4061-9e0d-60bdd60e1010 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:36.822938 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f606d0-bedb-4061-9e0d-60bdd60e1010 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:36.823213 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f606d0-bedb-4061-9e0d-60bdd60e1010 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:36.823972 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f606d0-bedb-4061-9e0d-60bdd60e1010 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:36.824281 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f606d0-bedb-4061-9e0d-60bdd60e1010 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:36.825671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f606d0-bedb-4061-9e0d-60bdd60e1010 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:36.828144 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:36.838240 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 258/509] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:36.839046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f606d0-bedb-4061-9e0d-60bdd60e1010 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:36.894754 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44c8a5-439e-4862-931c-6222237d06c2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:36.922004 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:36.931289 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f423ae-b9b6-45ec-84a3-f731ded97b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:36.934057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f423ae-b9b6-45ec-84a3-f731ded97b27 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:36.934057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f423ae-b9b6-45ec-84a3-f731ded97b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:36.934057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f423ae-b9b6-45ec-84a3-f731ded97b27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:36.939867 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f606d0-bedb-4061-9e0d-60bdd60e1010 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:36.943346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f423ae-b9b6-45ec-84a3-f731ded97b27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:36.945543 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f423ae-b9b6-45ec-84a3-f731ded97b27 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:36.963665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d8f423ae-b9b6-45ec-84a3-f731ded97b27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:36.963665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:36.963665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:36.963665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:36.963665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:36.963665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:36.963665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:36.963665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:36.963665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:36.963665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:36.963665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:36.963665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:36.963665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:36.963665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:36.963665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:36.964539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:36.964539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:36.964539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:36.964539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:36.964539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:36.964539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:36.964539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:36.964539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:36.964539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:36.964539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:36.964539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:36.964539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:36.964539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:36.964539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:36.964539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:36.964539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:36.964539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:36.965465 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:36.965465 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:36.965465 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:36.965465 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:36.965465 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 252/510] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:36.965772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f423ae-b9b6-45ec-84a3-f731ded97b27 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:36.980750 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452f0b2-7c1e-43f5-a1dd-0bb875a04968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:36.981727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452f0b2-7c1e-43f5-a1dd-0bb875a04968 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:36.981727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452f0b2-7c1e-43f5-a1dd-0bb875a04968 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:36.981957 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452f0b2-7c1e-43f5-a1dd-0bb875a04968 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:36.982799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e452f0b2-7c1e-43f5-a1dd-0bb875a04968 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:36.983109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e452f0b2-7c1e-43f5-a1dd-0bb875a04968 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:36.984599 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e452f0b2-7c1e-43f5-a1dd-0bb875a04968 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:37.022133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 259/511] 10.222.0.67 () {56 vars in 1137 bytes} [Thu Jun 30 16:13:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:37.023625 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452f0b2-7c1e-43f5-a1dd-0bb875a04968 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:37.075134 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f423ae-b9b6-45ec-84a3-f731ded97b27 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:37.078475 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:37.109146 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af052a-a492-4a6f-8c86-848671c95609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:37.109806 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af052a-a492-4a6f-8c86-848671c95609 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:37.110092 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af052a-a492-4a6f-8c86-848671c95609 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:37.110370 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af052a-a492-4a6f-8c86-848671c95609 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:37.111114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af052a-a492-4a6f-8c86-848671c95609 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:37.111423 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af052a-a492-4a6f-8c86-848671c95609 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:37.112844 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af052a-a492-4a6f-8c86-848671c95609 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:37.123762 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452f0b2-7c1e-43f5-a1dd-0bb875a04968 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:37.124686 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 253/512] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:37.125488 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af052a-a492-4a6f-8c86-848671c95609 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:37.162373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c63c8-779b-477b-876f-d3ae2268b914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:37.163116 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c63c8-779b-477b-876f-d3ae2268b914 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:37.163545 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c63c8-779b-477b-876f-d3ae2268b914 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:37.164027 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c63c8-779b-477b-876f-d3ae2268b914 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:37.167143 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:37.174666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c63c8-779b-477b-876f-d3ae2268b914 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:37.176180 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c63c8-779b-477b-876f-d3ae2268b914 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:37.193371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-7c4c63c8-779b-477b-876f-d3ae2268b914 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:37.193371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:37.193371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:37.193371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:37.193371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:37.193371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:37.193371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:37.193371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:37.193371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:37.193371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:37.193371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:37.193371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:37.193371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:37.193371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:37.193371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:37.194801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:37.194801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:37.194801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:37.194801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:37.194801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:37.194801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:37.194801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:37.194801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:37.194801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:37.194801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:37.194801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:37.194801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:37.194801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:37.194801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:37.194801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:37.194801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:37.194801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:37.195898 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:37.195898 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:37.195898 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:37.195898 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:37.195898 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 260/513] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:37.195898 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c63c8-779b-477b-876f-d3ae2268b914 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:37.236615 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af052a-a492-4a6f-8c86-848671c95609 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:37.271341 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb272e-9966-459b-b7bd-cb44d6ede058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:37.272162 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb272e-9966-459b-b7bd-cb44d6ede058 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:37.272457 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb272e-9966-459b-b7bd-cb44d6ede058 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:37.273330 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb272e-9966-459b-b7bd-cb44d6ede058 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:37.281184 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb272e-9966-459b-b7bd-cb44d6ede058 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:37.282623 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb272e-9966-459b-b7bd-cb44d6ede058 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:37.297250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-67fb272e-9966-459b-b7bd-cb44d6ede058 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 16:13:37.297250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:37.297250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:37.297250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:37.297250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:37.297250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:37.297250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:37.297250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:37.297250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:37.297250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:37.297250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:37.297250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:37.297250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:13:37.297250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:13:37.297250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:13:37.297250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:13:37.298226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:37.298226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:37.298226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:13:37.298226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:37.298226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:13:37.298226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:37.298226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:37.298226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:37.298226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:37.298226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:37.298226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:37.298226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:37.298226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:37.298226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:37.298226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:37.298226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:37.298226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:37.298226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:37.299122 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:37.299122 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:37.299122 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:13:37.299122 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:13:37.299122 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:13:37.299122 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:13:37.299122 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:13:37.299122 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:13:37.299122 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 16:13:37.299122 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:37.299122 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 254/514] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:37.299716 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb272e-9966-459b-b7bd-cb44d6ede058 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:37.305412 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c63c8-779b-477b-876f-d3ae2268b914 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:37.313185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:37.340285 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0284ed23-3c43-4919-9b8e-71a7594535f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:37.340979 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284ed23-3c43-4919-9b8e-71a7594535f0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:37.341267 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284ed23-3c43-4919-9b8e-71a7594535f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:37.341546 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284ed23-3c43-4919-9b8e-71a7594535f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:37.342274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284ed23-3c43-4919-9b8e-71a7594535f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:37.342583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284ed23-3c43-4919-9b8e-71a7594535f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:37.344155 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284ed23-3c43-4919-9b8e-71a7594535f0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:37.356417 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 261/515] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:37.357236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0284ed23-3c43-4919-9b8e-71a7594535f0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:37.396570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:37.401134 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb272e-9966-459b-b7bd-cb44d6ede058 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:37.436462 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567eb7be-63bd-4b38-9631-dbb23b495a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:37.437055 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567eb7be-63bd-4b38-9631-dbb23b495a2d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:37.437384 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567eb7be-63bd-4b38-9631-dbb23b495a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:37.437710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567eb7be-63bd-4b38-9631-dbb23b495a2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:37.463105 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0284ed23-3c43-4919-9b8e-71a7594535f0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:37.498865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778c844-718c-462c-9e14-8e9e29a49ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:37.499626 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778c844-718c-462c-9e14-8e9e29a49ecf None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:37.499914 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778c844-718c-462c-9e14-8e9e29a49ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:37.500185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c778c844-718c-462c-9e14-8e9e29a49ecf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:37.506207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567eb7be-63bd-4b38-9631-dbb23b495a2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:37.508152 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c778c844-718c-462c-9e14-8e9e29a49ecf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:37.508262 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567eb7be-63bd-4b38-9631-dbb23b495a2d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:37.509604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c778c844-718c-462c-9e14-8e9e29a49ecf None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:37.525774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-c778c844-718c-462c-9e14-8e9e29a49ecf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:37.525774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:37.525774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:37.525774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:37.525774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:37.525774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:37.525774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:37.525774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:37.525774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:37.525774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:37.525774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:37.525774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:37.525774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:13:37.525774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:13:37.525774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:13:37.526637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:13:37.526637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:37.526637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:37.526637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:37.526637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:37.526637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:37.526637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:37.526637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:37.526637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:37.526637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:37.526637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:37.526637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:37.526637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:37.526637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:37.526637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:37.526637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:37.526637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:37.527462 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:13:37.527462 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:13:37.527462 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:13:37.527462 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:13:37.527462 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:13:37.527462 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:13:37.527462 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:37.527462 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:37.527462 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 262/516] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:13:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:37.527939 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778c844-718c-462c-9e14-8e9e29a49ecf None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:37.555643 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:37.610862 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:37.615297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 255/517] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:13:37.616137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567eb7be-63bd-4b38-9631-dbb23b495a2d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:37.629838 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c778c844-718c-462c-9e14-8e9e29a49ecf None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:37.664998 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd94b6-f874-48af-b63b-cceb9b1ef72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:37.665762 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd94b6-f874-48af-b63b-cceb9b1ef72a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:37.666136 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd94b6-f874-48af-b63b-cceb9b1ef72a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:37.666426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd94b6-f874-48af-b63b-cceb9b1ef72a None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:37.667156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd94b6-f874-48af-b63b-cceb9b1ef72a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:37.667492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd94b6-f874-48af-b63b-cceb9b1ef72a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:37.668882 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd94b6-f874-48af-b63b-cceb9b1ef72a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:37.680841 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 263/518] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:37.680841 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd94b6-f874-48af-b63b-cceb9b1ef72a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:37.722818 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567eb7be-63bd-4b38-9631-dbb23b495a2d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:37.756386 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1087d38-3ede-4da3-9dfe-b314acc3c268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:37.757758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1087d38-3ede-4da3-9dfe-b314acc3c268 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:37.758043 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1087d38-3ede-4da3-9dfe-b314acc3c268 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:37.758319 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1087d38-3ede-4da3-9dfe-b314acc3c268 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:37.759148 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1087d38-3ede-4da3-9dfe-b314acc3c268 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:37.759456 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1087d38-3ede-4da3-9dfe-b314acc3c268 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:37.760927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1087d38-3ede-4da3-9dfe-b314acc3c268 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:37.768046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:37.779287 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 256/519] 10.222.0.67 () {56 vars in 1137 bytes} [Thu Jun 30 16:13:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:37.783159 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1087d38-3ede-4da3-9dfe-b314acc3c268 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:37.784279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd94b6-f874-48af-b63b-cceb9b1ef72a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:37.817130 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33143947-776d-41a2-9fa2-a398a924cbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:37.818066 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33143947-776d-41a2-9fa2-a398a924cbe0 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:37.818581 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33143947-776d-41a2-9fa2-a398a924cbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:37.818880 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33143947-776d-41a2-9fa2-a398a924cbe0 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:37.820625 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33143947-776d-41a2-9fa2-a398a924cbe0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:37.822070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33143947-776d-41a2-9fa2-a398a924cbe0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:37.836745 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:37.878207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1087d38-3ede-4da3-9dfe-b314acc3c268 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:37.913810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80bba5-dfad-47c9-931f-0452696b16d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:37.914467 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80bba5-dfad-47c9-931f-0452696b16d9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:37.914785 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80bba5-dfad-47c9-931f-0452696b16d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:37.915099 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d80bba5-dfad-47c9-931f-0452696b16d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:37.915488 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 264/520] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 30 16:13:37 2022] POST /identity/v3/users => generated 247 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:13:37.920650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80bba5-dfad-47c9-931f-0452696b16d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:37.920650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80bba5-dfad-47c9-931f-0452696b16d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:37.920876 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33143947-776d-41a2-9fa2-a398a924cbe0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:37.920956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d80bba5-dfad-47c9-931f-0452696b16d9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:37.938001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 257/521] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:13:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:37.938689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80bba5-dfad-47c9-931f-0452696b16d9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:38.062064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33143947-776d-41a2-9fa2-a398a924cbe0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:38.063377 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d80bba5-dfad-47c9-931f-0452696b16d9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:38.081190 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:38.101380 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a418696-0c25-413e-80b4-9ae7b7d20c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:38.102188 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a418696-0c25-413e-80b4-9ae7b7d20c95 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:38.102469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a418696-0c25-413e-80b4-9ae7b7d20c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:38.102745 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a418696-0c25-413e-80b4-9ae7b7d20c95 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:38.107429 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f37559-47d4-4028-9ae8-05f5741fcc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:38.108574 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f37559-47d4-4028-9ae8-05f5741fcc73 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:38.108837 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f37559-47d4-4028-9ae8-05f5741fcc73 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:38.109116 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f37559-47d4-4028-9ae8-05f5741fcc73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:38.110466 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a418696-0c25-413e-80b4-9ae7b7d20c95 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:38.112049 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a418696-0c25-413e-80b4-9ae7b7d20c95 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:38.124203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f37559-47d4-4028-9ae8-05f5741fcc73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:38.126327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f37559-47d4-4028-9ae8-05f5741fcc73 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:38.129191 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:38.131771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-8a418696-0c25-413e-80b4-9ae7b7d20c95 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 16:13:38.131771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:38.131771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:38.131771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:38.131771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:38.131771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:38.131771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:38.131771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:38.131771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:38.131771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:38.131771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:38.131771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:38.131771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:13:38.131771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:13:38.131771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:13:38.131771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:13:38.132803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:38.132803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:38.132803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:13:38.132803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:38.132803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:13:38.132803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:38.132803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:38.132803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:38.132803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:38.132803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:38.132803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:38.132803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:38.132803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:38.132803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:38.132803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:38.132803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:38.132803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:38.132803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:38.133677 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:38.133677 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:38.133677 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:13:38.133677 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:13:38.133677 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:13:38.133677 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:13:38.133677 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:13:38.133677 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:13:38.133677 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 16:13:38.133677 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:38.133677 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 258/522] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 30 16:13:37 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:38.134709 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a418696-0c25-413e-80b4-9ae7b7d20c95 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:38.145236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-26f37559-47d4-4028-9ae8-05f5741fcc73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:38.145236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:38.145236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:38.145236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:38.145236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:38.145236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:38.145236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:38.145236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:38.145236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:38.145236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:38.145236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:38.145236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:38.145236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:38.145236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:38.145236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:38.146343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:38.146343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:38.146343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:38.146343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:38.146343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:38.146343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:38.146343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:38.146343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:38.146343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:38.146343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:38.146343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:38.146343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:38.146343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:38.146343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:38.146343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:38.146343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:38.146343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:38.147268 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:38.147268 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:38.147268 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:38.147268 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:38.147268 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 265/523] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:38.148392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f37559-47d4-4028-9ae8-05f5741fcc73 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:38.255647 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f37559-47d4-4028-9ae8-05f5741fcc73 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:38.262428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a418696-0c25-413e-80b4-9ae7b7d20c95 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:38.295486 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da82c2c-6582-49c0-a098-7d24acae612f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:38.296157 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da82c2c-6582-49c0-a098-7d24acae612f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:38.296459 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da82c2c-6582-49c0-a098-7d24acae612f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:38.296711 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da82c2c-6582-49c0-a098-7d24acae612f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:38.297596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da82c2c-6582-49c0-a098-7d24acae612f None admin] RBAC: Adding query filter params (role.id=57bc5df632b3407d97116ae34d92a5cb, scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d, user.id=71cca3708386459197abf6c67505bc5f) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:38.297898 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da82c2c-6582-49c0-a098-7d24acae612f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:38.299355 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da82c2c-6582-49c0-a098-7d24acae612f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:38.300190 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1003e8-e3ec-429c-a6d4-7b95782a4831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:38.300838 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1003e8-e3ec-429c-a6d4-7b95782a4831 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:38.301330 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1003e8-e3ec-429c-a6d4-7b95782a4831 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:38.301613 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1003e8-e3ec-429c-a6d4-7b95782a4831 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:38.316977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 266/524] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:13:38 2022] GET /identity/v3/role_assignments?user.id=71cca3708386459197abf6c67505bc5f&scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d&role.id=57bc5df632b3407d97116ae34d92a5cb => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:38.316977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da82c2c-6582-49c0-a098-7d24acae612f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:38.365937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:38.391077 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:38.402123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1003e8-e3ec-429c-a6d4-7b95782a4831 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:38.405720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1003e8-e3ec-429c-a6d4-7b95782a4831 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:38.447430 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da82c2c-6582-49c0-a098-7d24acae612f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:38.498813 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4dd173-bfc6-4359-a79d-ee73fc145673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:38.499968 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4dd173-bfc6-4359-a79d-ee73fc145673 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:38.500191 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4dd173-bfc6-4359-a79d-ee73fc145673 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:38.500470 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4dd173-bfc6-4359-a79d-ee73fc145673 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:38.501357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4dd173-bfc6-4359-a79d-ee73fc145673 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:38.501672 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4dd173-bfc6-4359-a79d-ee73fc145673 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:38.503155 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4dd173-bfc6-4359-a79d-ee73fc145673 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:38.526614 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 267/525] 10.222.0.67 () {56 vars in 1139 bytes} [Thu Jun 30 16:13:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:38.527553 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4dd173-bfc6-4359-a79d-ee73fc145673 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:38.537339 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 259/526] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:13:38.538239 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1003e8-e3ec-429c-a6d4-7b95782a4831 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:38.619455 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:38.643745 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1003e8-e3ec-429c-a6d4-7b95782a4831 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:38.659863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4dd173-bfc6-4359-a79d-ee73fc145673 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:38.686077 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c20aaf-4bea-4390-8b3e-7fd6a5e2fecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:38.687021 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c20aaf-4bea-4390-8b3e-7fd6a5e2fecd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:38.687456 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c20aaf-4bea-4390-8b3e-7fd6a5e2fecd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:38.687914 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c20aaf-4bea-4390-8b3e-7fd6a5e2fecd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:38.703859 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:38.706663 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b437b-f8af-4518-a3a3-61b739f2d57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:38.707347 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b437b-f8af-4518-a3a3-61b739f2d57b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:38.707690 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b437b-f8af-4518-a3a3-61b739f2d57b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:38.708042 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b437b-f8af-4518-a3a3-61b739f2d57b None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:38.708811 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1b437b-f8af-4518-a3a3-61b739f2d57b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:38.709643 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1b437b-f8af-4518-a3a3-61b739f2d57b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:38.712233 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1b437b-f8af-4518-a3a3-61b739f2d57b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:38.731578 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 268/527] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:38.732741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b437b-f8af-4518-a3a3-61b739f2d57b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:38.790899 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c20aaf-4bea-4390-8b3e-7fd6a5e2fecd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:38.792709 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c20aaf-4bea-4390-8b3e-7fd6a5e2fecd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:38.846129 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b437b-f8af-4518-a3a3-61b739f2d57b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:38.894450 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53818bce-dd15-4b05-a609-8855bb623df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:38.895423 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53818bce-dd15-4b05-a609-8855bb623df4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:38.895839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53818bce-dd15-4b05-a609-8855bb623df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:38.896218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53818bce-dd15-4b05-a609-8855bb623df4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:38.897444 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:38.913347 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53818bce-dd15-4b05-a609-8855bb623df4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:38.915630 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53818bce-dd15-4b05-a609-8855bb623df4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:38.926342 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 260/528] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:13:38.927622 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c20aaf-4bea-4390-8b3e-7fd6a5e2fecd None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:38.939209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-53818bce-dd15-4b05-a609-8855bb623df4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:38.939209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:38.939209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:38.939209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:38.939209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:38.939209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:38.939209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:38.939209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:38.939209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:38.939209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:38.939209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:38.939209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:38.939209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:38.939209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:38.939209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:38.940411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:38.940411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:38.940411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:38.940411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:38.940411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:38.940411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:38.940411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:38.940411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:38.940411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:38.940411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:38.940411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:38.940411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:38.940411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:38.940411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:38.940411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:38.940411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:38.940411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:38.941460 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:38.941460 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:38.941460 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:38.941460 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:38.947559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 269/529] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:38.948705 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53818bce-dd15-4b05-a609-8855bb623df4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:38.986650 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:39.057997 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c20aaf-4bea-4390-8b3e-7fd6a5e2fecd None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:39.081475 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53818bce-dd15-4b05-a609-8855bb623df4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:39.104745 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f118014-dc8a-41e1-8bf5-e7573ff68541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:39.106363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f118014-dc8a-41e1-8bf5-e7573ff68541 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:39.106757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f118014-dc8a-41e1-8bf5-e7573ff68541 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:39.107135 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f118014-dc8a-41e1-8bf5-e7573ff68541 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:39.117655 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f118014-dc8a-41e1-8bf5-e7573ff68541 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:39.119567 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f118014-dc8a-41e1-8bf5-e7573ff68541 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:39.122161 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04231245-237b-491e-9c15-219b3ae0bf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:39.123297 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04231245-237b-491e-9c15-219b3ae0bf5f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:39.123755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04231245-237b-491e-9c15-219b3ae0bf5f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:39.124200 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04231245-237b-491e-9c15-219b3ae0bf5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:39.135982 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04231245-237b-491e-9c15-219b3ae0bf5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:39.137477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-4f118014-dc8a-41e1-8bf5-e7573ff68541 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:39.137477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:39.137477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:39.137477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:39.137477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:39.137477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:39.137477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:39.137477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:39.137477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:39.137477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:39.137477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:39.137477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:39.137477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:13:39.137477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:13:39.137477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:13:39.138491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:13:39.138491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:39.138491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:39.138491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:39.138491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:39.138491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:39.138491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:39.138491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:39.138491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:39.138491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:39.138491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:39.138491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:39.138491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:39.138491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:39.138491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:39.138491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:39.138491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:39.139598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:13:39.139598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:13:39.139598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:13:39.139598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:13:39.139598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:13:39.139598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:13:39.139598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:39.139598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:39.140378 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04231245-237b-491e-9c15-219b3ae0bf5f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:39.141215 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 261/530] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:13:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:39.142092 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f118014-dc8a-41e1-8bf5-e7573ff68541 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:39.162203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-04231245-237b-491e-9c15-219b3ae0bf5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:39.162203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:39.162203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:39.162203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:39.162203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:39.162203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:39.162203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:39.162203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:39.162203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:39.162203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:39.162203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:39.162203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:39.162203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:39.162203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:39.162203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:39.163306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:39.163306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:39.163306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:39.163306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:39.163306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:39.163306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:39.163306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:39.163306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:39.163306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:39.163306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:39.163306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:39.163306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:39.163306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:39.163306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:39.163306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:39.163306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:39.163306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:39.164388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:39.164388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:39.164388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:39.164388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:39.166640 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 270/531] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:39.167713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04231245-237b-491e-9c15-219b3ae0bf5f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:39.188533 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:39.265151 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:39.271937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f118014-dc8a-41e1-8bf5-e7573ff68541 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:39.287266 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04231245-237b-491e-9c15-219b3ae0bf5f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:39.316452 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf887c9-03e2-4bf0-95f5-8079a2915243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:39.317355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf887c9-03e2-4bf0-95f5-8079a2915243 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:39.317706 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf887c9-03e2-4bf0-95f5-8079a2915243 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:39.318046 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf887c9-03e2-4bf0-95f5-8079a2915243 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:39.319139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf887c9-03e2-4bf0-95f5-8079a2915243 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:39.320375 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf887c9-03e2-4bf0-95f5-8079a2915243 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:39.321933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf887c9-03e2-4bf0-95f5-8079a2915243 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:39.324787 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a90a0-2ded-4056-880a-c442e42fcf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:39.325647 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a90a0-2ded-4056-880a-c442e42fcf14 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:39.326115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a90a0-2ded-4056-880a-c442e42fcf14 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:39.326588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a90a0-2ded-4056-880a-c442e42fcf14 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:39.327582 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a90a0-2ded-4056-880a-c442e42fcf14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:39.328158 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a90a0-2ded-4056-880a-c442e42fcf14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:39.329912 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a90a0-2ded-4056-880a-c442e42fcf14 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:39.340043 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 262/532] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:39.341000 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf887c9-03e2-4bf0-95f5-8079a2915243 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:39.355546 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 271/533] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:13:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:39.356766 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a90a0-2ded-4056-880a-c442e42fcf14 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:39.457592 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:39.465792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf887c9-03e2-4bf0-95f5-8079a2915243 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:39.490979 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a90a0-2ded-4056-880a-c442e42fcf14 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:39.511923 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6aa550-69c9-415c-a554-9d118f6576b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:39.512607 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6aa550-69c9-415c-a554-9d118f6576b2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:39.512892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6aa550-69c9-415c-a554-9d118f6576b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:39.513170 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6aa550-69c9-415c-a554-9d118f6576b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:39.513912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6aa550-69c9-415c-a554-9d118f6576b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:39.514251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6aa550-69c9-415c-a554-9d118f6576b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:39.515782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6aa550-69c9-415c-a554-9d118f6576b2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:39.529869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 263/534] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:39.530732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6aa550-69c9-415c-a554-9d118f6576b2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:39.538310 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd51a96-a74c-452c-b23e-d884585f3289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:39.539398 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd51a96-a74c-452c-b23e-d884585f3289 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:39.539810 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd51a96-a74c-452c-b23e-d884585f3289 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:39.540185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd51a96-a74c-452c-b23e-d884585f3289 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:39.549257 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd51a96-a74c-452c-b23e-d884585f3289 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:39.551584 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd51a96-a74c-452c-b23e-d884585f3289 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:39.555571 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:39.572563 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-9cd51a96-a74c-452c-b23e-d884585f3289 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:39.572563 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:39.572563 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:39.572563 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:39.572563 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:39.572563 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:39.572563 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:39.572563 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:39.572563 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:39.572563 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:39.572563 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:39.572563 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:39.572563 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:13:39.572563 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:13:39.572563 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:13:39.573484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:13:39.573484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:39.573484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:39.573484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:39.573484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:39.573484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:39.573484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:39.573484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:39.573484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:39.573484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:39.573484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:39.573484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:39.573484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:39.573484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:39.573484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:39.573484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:39.573484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:39.574299 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:13:39.574299 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:13:39.574299 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:13:39.574299 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:13:39.574299 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:13:39.574299 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:13:39.574299 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:39.574299 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:39.581935 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 272/535] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:13:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:39.581935 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd51a96-a74c-452c-b23e-d884585f3289 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:39.668728 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6aa550-69c9-415c-a554-9d118f6576b2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:39.700001 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd51a96-a74c-452c-b23e-d884585f3289 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:39.710702 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3eb35-5f95-4109-9cd5-abd2adb6b7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:39.711410 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3eb35-5f95-4109-9cd5-abd2adb6b7ac None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:39.711780 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3eb35-5f95-4109-9cd5-abd2adb6b7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:39.712061 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3eb35-5f95-4109-9cd5-abd2adb6b7ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:39.713124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3eb35-5f95-4109-9cd5-abd2adb6b7ac None admin] RBAC: Adding query filter params (role.id=57bc5df632b3407d97116ae34d92a5cb, scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d, user.id=61437a2fa2a94b35b1b4a5750f6e2cad) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:39.713322 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3eb35-5f95-4109-9cd5-abd2adb6b7ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:39.714904 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3eb35-5f95-4109-9cd5-abd2adb6b7ac None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:39.727814 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 264/536] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:13:39 2022] GET /identity/v3/role_assignments?user.id=61437a2fa2a94b35b1b4a5750f6e2cad&scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d&role.id=57bc5df632b3407d97116ae34d92a5cb => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:39.728849 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3eb35-5f95-4109-9cd5-abd2adb6b7ac None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:39.743637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e30038-036f-4380-b061-df130a876c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:39.744804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e30038-036f-4380-b061-df130a876c0c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:39.745285 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e30038-036f-4380-b061-df130a876c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:39.746522 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e30038-036f-4380-b061-df130a876c0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:39.764225 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e30038-036f-4380-b061-df130a876c0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:39.769572 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e30038-036f-4380-b061-df130a876c0c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:39.782407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:39.796045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-a6e30038-036f-4380-b061-df130a876c0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:39.796045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:39.796045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:39.796045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:39.796045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:39.796045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:39.796045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:39.796045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:39.796045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:39.796045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:39.796045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:39.796045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:39.796045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:13:39.796045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:13:39.796045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:13:39.796969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:13:39.796969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:39.796969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:39.796969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:39.796969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:39.796969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:39.796969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:39.796969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:39.796969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:39.796969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:39.796969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:39.796969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:39.796969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:39.796969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:39.796969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:39.796969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:39.796969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:39.797795 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:13:39.797795 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:13:39.797795 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:13:39.797795 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:13:39.797795 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:13:39.797795 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:13:39.797795 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:39.797795 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:39.812110 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 273/537] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:13:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:39.820773 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e30038-036f-4380-b061-df130a876c0c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:39.900275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3eb35-5f95-4109-9cd5-abd2adb6b7ac None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:39.946438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:39.962315 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4834a-dd01-42bb-93f2-dd9cfe85128c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:39.963339 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4834a-dd01-42bb-93f2-dd9cfe85128c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:39.963847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4834a-dd01-42bb-93f2-dd9cfe85128c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:39.964346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4834a-dd01-42bb-93f2-dd9cfe85128c None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:39.965340 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4834a-dd01-42bb-93f2-dd9cfe85128c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:39.967108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4834a-dd01-42bb-93f2-dd9cfe85128c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:39.968831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4834a-dd01-42bb-93f2-dd9cfe85128c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:40.018415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 265/538] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:13:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:40.029489 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4834a-dd01-42bb-93f2-dd9cfe85128c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:40.045977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e30038-036f-4380-b061-df130a876c0c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:40.113704 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f269f0-b4fe-496d-83fb-67455e55d268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:40.114425 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f269f0-b4fe-496d-83fb-67455e55d268 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:40.114708 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f269f0-b4fe-496d-83fb-67455e55d268 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:40.114983 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f269f0-b4fe-496d-83fb-67455e55d268 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:40.115810 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f269f0-b4fe-496d-83fb-67455e55d268 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:40.116658 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f269f0-b4fe-496d-83fb-67455e55d268 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:40.118150 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f269f0-b4fe-496d-83fb-67455e55d268 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:40.134044 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 274/539] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:13:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:40.135885 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba760e53-12c7-4b24-97de-f292c8c7060b None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:40.136359 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba760e53-12c7-4b24-97de-f292c8c7060b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:40.136814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba760e53-12c7-4b24-97de-f292c8c7060b None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:40.142796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 275/540] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:40.143914 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba760e53-12c7-4b24-97de-f292c8c7060b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:40.165373 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:40.190051 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4834a-dd01-42bb-93f2-dd9cfe85128c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:40.238667 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c52d2d-4b54-42f1-aaf4-521acb385ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:40.239766 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c52d2d-4b54-42f1-aaf4-521acb385ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:40.240156 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c52d2d-4b54-42f1-aaf4-521acb385ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:40.240604 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c52d2d-4b54-42f1-aaf4-521acb385ae7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:40.242869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c52d2d-4b54-42f1-aaf4-521acb385ae7 None admin] RBAC: Adding query filter params (role.id=57bc5df632b3407d97116ae34d92a5cb, scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d, user.id=ec41c84c9375437590c42621ca5b400a) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:40.243741 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c52d2d-4b54-42f1-aaf4-521acb385ae7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:40.246413 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c52d2d-4b54-42f1-aaf4-521acb385ae7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:40.255580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 266/541] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:13:40 2022] GET /identity/v3/role_assignments?user.id=ec41c84c9375437590c42621ca5b400a&scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d&role.id=57bc5df632b3407d97116ae34d92a5cb => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:40.257183 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe83d1e-73d0-4b10-acea-7899c7d017ba None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:40.257608 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe83d1e-73d0-4b10-acea-7899c7d017ba None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:40.258045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe83d1e-73d0-4b10-acea-7899c7d017ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:40.271821 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba760e53-12c7-4b24-97de-f292c8c7060b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:40.296686 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-dfe83d1e-73d0-4b10-acea-7899c7d017ba None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:40.298285 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dfe83d1e-73d0-4b10-acea-7899c7d017ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:40.301893 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:40.340192 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5620fa8-88bf-48ed-a69b-1d6e29fbaaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:40.341064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5620fa8-88bf-48ed-a69b-1d6e29fbaaae None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:40.341449 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5620fa8-88bf-48ed-a69b-1d6e29fbaaae None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:40.341825 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5620fa8-88bf-48ed-a69b-1d6e29fbaaae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:40.342837 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5620fa8-88bf-48ed-a69b-1d6e29fbaaae None admin] RBAC: Adding query filter params (role.id=57bc5df632b3407d97116ae34d92a5cb, scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d, user.id=9b508dd8b7244d5f979e1ce7787d2c67) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:40.344726 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5620fa8-88bf-48ed-a69b-1d6e29fbaaae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:40.347217 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5620fa8-88bf-48ed-a69b-1d6e29fbaaae None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:40.361392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 276/542] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:13:40 2022] GET /identity/v3/role_assignments?user.id=9b508dd8b7244d5f979e1ce7787d2c67&scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d&role.id=57bc5df632b3407d97116ae34d92a5cb => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:40.429847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 267/543] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:40.455408 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5620fa8-88bf-48ed-a69b-1d6e29fbaaae None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:40.555494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03229b28-eba2-460c-af5c-e3274d8df4c0 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:40.555999 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03229b28-eba2-460c-af5c-e3274d8df4c0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:40.556071 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03229b28-eba2-460c-af5c-e3274d8df4c0 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:40.557574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 268/544] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:13:40.563164 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9321b-44c8-43d9-90d3-430660e7da76 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:40.563450 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9321b-44c8-43d9-90d3-430660e7da76 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:40.563751 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9321b-44c8-43d9-90d3-430660e7da76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:40.592573 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-81b9321b-44c8-43d9-90d3-430660e7da76 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:40.593787 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-81b9321b-44c8-43d9-90d3-430660e7da76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:40.629213 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5620fa8-88bf-48ed-a69b-1d6e29fbaaae None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:40.665448 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6722ae-007c-4f78-aaad-7d2a46106025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:40.666250 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6722ae-007c-4f78-aaad-7d2a46106025 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:40.666549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6722ae-007c-4f78-aaad-7d2a46106025 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:40.666818 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6722ae-007c-4f78-aaad-7d2a46106025 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:40.678010 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6722ae-007c-4f78-aaad-7d2a46106025 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:40.687396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-3d6722ae-007c-4f78-aaad-7d2a46106025 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:40.687396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:40.687396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:40.687396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:40.687396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:40.687396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:40.687396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:40.687396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:40.687396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:40.687396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:40.687396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:40.687396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:40.687396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:40.687396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:40.687396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:40.687396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:40.688332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:40.688332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:40.688332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:40.688332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:40.688332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:40.688332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:40.688332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:40.688332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:40.688332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:40.688332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:40.688332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:40.688332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:40.688332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:40.688332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:40.688332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:40.688332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:40.689554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:40.689554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:40.689554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:40.689554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:40.689554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:40.689554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:40.689554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:40.689554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:40.689554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:40.689554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:40.689554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:40.689554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:40.689554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 277/545] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 30 16:13:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:40.695087 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6722ae-007c-4f78-aaad-7d2a46106025 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:40.742170 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 269/546] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 16:13:40.775933 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:40.779959 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b9321b-44c8-43d9-90d3-430660e7da76 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:40.783795 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:40.873680 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6722ae-007c-4f78-aaad-7d2a46106025 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:40.941999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9d347-3ba7-42de-a4b4-dbbf7a3132ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:40.942948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9d347-3ba7-42de-a4b4-dbbf7a3132ce None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:40.943652 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9d347-3ba7-42de-a4b4-dbbf7a3132ce None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:40.945529 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9d347-3ba7-42de-a4b4-dbbf7a3132ce None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:40.946443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9d347-3ba7-42de-a4b4-dbbf7a3132ce None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:40.947142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9d347-3ba7-42de-a4b4-dbbf7a3132ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:40.949774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9d347-3ba7-42de-a4b4-dbbf7a3132ce None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:40.968552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 278/547] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 30 16:13:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:40.974228 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9d347-3ba7-42de-a4b4-dbbf7a3132ce None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:40.998869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b9321b-44c8-43d9-90d3-430660e7da76 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:41.064874 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6b53e-df01-4c8b-bfa4-6045996bf9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:41.068002 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6b53e-df01-4c8b-bfa4-6045996bf9ee None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:41.068438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6b53e-df01-4c8b-bfa4-6045996bf9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:41.068880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c6b53e-df01-4c8b-bfa4-6045996bf9ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:41.095329 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c6b53e-df01-4c8b-bfa4-6045996bf9ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:41.107693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-47c6b53e-df01-4c8b-bfa4-6045996bf9ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:41.107693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:41.107693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:41.107693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:41.107693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:41.107693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:41.107693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:41.107693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:41.107693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:41.107693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:41.107693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:41.107693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:41.107693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:41.107693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:41.107693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:41.107693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:41.108869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:41.108869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:41.108869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:41.108869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:41.108869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:41.108869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:41.108869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:41.108869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:41.108869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:41.108869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:41.108869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:41.108869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:41.108869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:41.108869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:41.108869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:41.108869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:41.110203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:41.110203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:41.110203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:41.110203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:41.110203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:41.110203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:41.110203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:41.110203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:41.110203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:41.110203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:41.110203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:41.110203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:41.110203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 270/548] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 30 16:13:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:41.115372 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6b53e-df01-4c8b-bfa4-6045996bf9ee None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:41.165254 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9d347-3ba7-42de-a4b4-dbbf7a3132ce None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:41.179929 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:41.215954 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:41.236753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b628789-7ecc-42b4-9b51-2e8abcf5e19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:41.237570 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b628789-7ecc-42b4-9b51-2e8abcf5e19e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:41.237877 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b628789-7ecc-42b4-9b51-2e8abcf5e19e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:41.238154 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b628789-7ecc-42b4-9b51-2e8abcf5e19e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:41.252724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c6b53e-df01-4c8b-bfa4-6045996bf9ee None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:41.265364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b628789-7ecc-42b4-9b51-2e8abcf5e19e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:41.266909 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b628789-7ecc-42b4-9b51-2e8abcf5e19e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:41.289182 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d26a60-a1d1-43fc-9d04-4ac3cb0b4c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:41.289978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d26a60-a1d1-43fc-9d04-4ac3cb0b4c94 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:41.290362 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d26a60-a1d1-43fc-9d04-4ac3cb0b4c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:41.290740 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d26a60-a1d1-43fc-9d04-4ac3cb0b4c94 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:41.291616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d26a60-a1d1-43fc-9d04-4ac3cb0b4c94 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:41.292044 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d26a60-a1d1-43fc-9d04-4ac3cb0b4c94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:41.293526 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d26a60-a1d1-43fc-9d04-4ac3cb0b4c94 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:41.296364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-4b628789-7ecc-42b4-9b51-2e8abcf5e19e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:41.296364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:41.296364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:41.296364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:41.296364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:41.296364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:41.296364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:41.296364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:41.296364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:41.296364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:41.296364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:41.296364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:41.296364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:41.296364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:41.296364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:41.297236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:41.297236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:41.297236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:41.297236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:41.297236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:41.297236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:41.297236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:41.297236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:41.297236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:41.297236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:41.297236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:41.297236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:41.297236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:41.297236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:41.297236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:41.297236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:41.297236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:41.298270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:41.298270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:41.298270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:41.298270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:41.298270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 279/549] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:13:41.304776 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b628789-7ecc-42b4-9b51-2e8abcf5e19e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:41.307923 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 271/550] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 30 16:13:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:41.316462 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d26a60-a1d1-43fc-9d04-4ac3cb0b4c94 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:41.429810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d26a60-a1d1-43fc-9d04-4ac3cb0b4c94 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:41.456468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:41.465536 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082d2bb-981d-4047-924b-dc0abcccaa47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:41.466487 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082d2bb-981d-4047-924b-dc0abcccaa47 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:41.466722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082d2bb-981d-4047-924b-dc0abcccaa47 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:41.466994 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082d2bb-981d-4047-924b-dc0abcccaa47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:41.468715 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b628789-7ecc-42b4-9b51-2e8abcf5e19e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:41.478791 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9082d2bb-981d-4047-924b-dc0abcccaa47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:41.480278 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9082d2bb-981d-4047-924b-dc0abcccaa47 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:41.498663 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9082d2bb-981d-4047-924b-dc0abcccaa47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:41.498663 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:41.498663 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:41.498663 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:41.498663 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:41.498663 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:41.498663 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:41.498663 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:41.498663 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:41.498663 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:41.498663 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:41.498663 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:41.498663 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:41.498663 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:41.498663 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:41.500597 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:41.500597 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:41.500597 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:41.500597 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:41.500597 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:41.500597 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:41.500597 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:41.500597 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:41.500597 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:41.500597 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:41.500597 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:41.500597 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:41.500597 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:41.500597 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:41.500597 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:41.500597 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:41.500597 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:41.501682 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:41.501682 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:41.501682 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:41.501682 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:41.501682 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 272/551] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:41.510028 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082d2bb-981d-4047-924b-dc0abcccaa47 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:41.534048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9cd9bc-96cf-4ac2-9313-f4c528513b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:41.534983 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cd9bc-96cf-4ac2-9313-f4c528513b5d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:41.535505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cd9bc-96cf-4ac2-9313-f4c528513b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:41.535934 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9cd9bc-96cf-4ac2-9313-f4c528513b5d None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:41.536942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9cd9bc-96cf-4ac2-9313-f4c528513b5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:41.537395 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9cd9bc-96cf-4ac2-9313-f4c528513b5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:41.539626 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9cd9bc-96cf-4ac2-9313-f4c528513b5d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:41.556915 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 280/552] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:41.563059 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9cd9bc-96cf-4ac2-9313-f4c528513b5d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:41.601857 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:41.651606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082d2bb-981d-4047-924b-dc0abcccaa47 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:41.692080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd59f2-98c3-4a27-a387-58a20f0b98e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:41.693287 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd59f2-98c3-4a27-a387-58a20f0b98e5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:41.694844 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd59f2-98c3-4a27-a387-58a20f0b98e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:41.695464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd59f2-98c3-4a27-a387-58a20f0b98e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:41.696988 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd59f2-98c3-4a27-a387-58a20f0b98e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:41.697727 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd59f2-98c3-4a27-a387-58a20f0b98e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:41.699899 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd59f2-98c3-4a27-a387-58a20f0b98e5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:41.727519 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 273/553] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:41.735570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd59f2-98c3-4a27-a387-58a20f0b98e5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:41.771504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9cd9bc-96cf-4ac2-9313-f4c528513b5d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:41.820199 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da724e75-2795-4c51-91dd-8210bb921242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:41.821018 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da724e75-2795-4c51-91dd-8210bb921242 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:41.821305 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da724e75-2795-4c51-91dd-8210bb921242 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:41.821581 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da724e75-2795-4c51-91dd-8210bb921242 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:41.847573 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da724e75-2795-4c51-91dd-8210bb921242 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:41.849147 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da724e75-2795-4c51-91dd-8210bb921242 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:41.871837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:41.882944 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd59f2-98c3-4a27-a387-58a20f0b98e5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:41.886124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-da724e75-2795-4c51-91dd-8210bb921242 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 16:13:41.886124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:41.886124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:41.886124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:41.886124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:41.886124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:41.886124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:41.886124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:41.886124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:41.886124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:41.886124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:41.886124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:41.886124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:13:41.886124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:13:41.886124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:13:41.886124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:13:41.886994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:41.886994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:41.886994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:13:41.886994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:41.886994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:13:41.886994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:41.886994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:41.886994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:41.886994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:41.886994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:41.886994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:41.886994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:41.886994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:41.886994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:41.886994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:41.886994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:41.886994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:41.886994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:41.887921 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:41.887921 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:41.887921 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:13:41.887921 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:13:41.887921 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:13:41.887921 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:13:41.887921 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:13:41.887921 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:13:41.887921 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 16:13:41.887921 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:41.893089 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 281/554] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:13:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:41.893949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da724e75-2795-4c51-91dd-8210bb921242 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:41.956187 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c1c8e-719e-4edc-9085-5e862bffbbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:41.957002 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c1c8e-719e-4edc-9085-5e862bffbbe3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:41.957291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c1c8e-719e-4edc-9085-5e862bffbbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:41.957982 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6c1c8e-719e-4edc-9085-5e862bffbbe3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:41.977963 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c1c8e-719e-4edc-9085-5e862bffbbe3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:41.979359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6c1c8e-719e-4edc-9085-5e862bffbbe3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:42.001559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-cd6c1c8e-719e-4edc-9085-5e862bffbbe3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 16:13:42.001559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:42.001559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:42.001559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:42.001559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:42.001559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:42.001559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:42.001559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:42.001559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:42.001559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:42.001559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:42.001559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:42.001559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:13:42.001559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:13:42.001559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:13:42.001559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:13:42.003675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:42.003675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:42.003675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:13:42.003675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:42.003675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:13:42.003675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:42.003675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:42.003675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:42.003675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:42.003675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:42.003675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:42.003675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:42.003675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:42.003675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:42.003675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:42.003675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:42.003675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:42.003675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:42.006279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:42.006279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:42.006279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:13:42.006279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:13:42.006279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:13:42.006279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:13:42.006279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:13:42.006279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:13:42.006279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 16:13:42.006279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:42.006279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 274/555] 10.222.0.67 () {56 vars in 1093 bytes} [Thu Jun 30 16:13:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:42.021993 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c1c8e-719e-4edc-9085-5e862bffbbe3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:42.043650 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:42.092467 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da724e75-2795-4c51-91dd-8210bb921242 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:42.154247 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df07e7-9fa3-47e3-8e7c-6b241bcb3604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:42.154933 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df07e7-9fa3-47e3-8e7c-6b241bcb3604 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:42.155221 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df07e7-9fa3-47e3-8e7c-6b241bcb3604 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:42.155512 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df07e7-9fa3-47e3-8e7c-6b241bcb3604 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:42.164683 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df07e7-9fa3-47e3-8e7c-6b241bcb3604 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:42.165015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df07e7-9fa3-47e3-8e7c-6b241bcb3604 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:42.166607 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df07e7-9fa3-47e3-8e7c-6b241bcb3604 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:42.196613 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 282/556] 10.222.0.67 () {56 vars in 1135 bytes} [Thu Jun 30 16:13:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:42.201410 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:42.201541 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87955d9f-974e-4180-8d27-a95a1ce76a96 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:42.201541 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87955d9f-974e-4180-8d27-a95a1ce76a96 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:42.201541 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87955d9f-974e-4180-8d27-a95a1ce76a96 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:42.201541 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 283/557] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:42.203584 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87955d9f-974e-4180-8d27-a95a1ce76a96 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:42.205693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6c1c8e-719e-4edc-9085-5e862bffbbe3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:42.244755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b561cfd-18f5-421f-ad6d-a7c27be2d596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:42.245742 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b561cfd-18f5-421f-ad6d-a7c27be2d596 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:42.246208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b561cfd-18f5-421f-ad6d-a7c27be2d596 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:42.246799 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b561cfd-18f5-421f-ad6d-a7c27be2d596 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:42.248673 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b561cfd-18f5-421f-ad6d-a7c27be2d596 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:42.249208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b561cfd-18f5-421f-ad6d-a7c27be2d596 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:42.250838 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b561cfd-18f5-421f-ad6d-a7c27be2d596 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:42.274809 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 275/558] 10.222.0.67 () {56 vars in 1133 bytes} [Thu Jun 30 16:13:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:42.276681 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfbe45-a7b1-42c8-866a-f3d090f9bc31 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:42.276967 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfbe45-a7b1-42c8-866a-f3d090f9bc31 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:42.277241 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfbe45-a7b1-42c8-866a-f3d090f9bc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:42.299219 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-eecfbe45-a7b1-42c8-866a-f3d090f9bc31 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:42.301099 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-eecfbe45-a7b1-42c8-866a-f3d090f9bc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:42.323023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87955d9f-974e-4180-8d27-a95a1ce76a96 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:42.377574 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccbcf4-4dab-493d-9398-918eb0956ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:42.378343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccbcf4-4dab-493d-9398-918eb0956ebd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:42.378699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccbcf4-4dab-493d-9398-918eb0956ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:42.379050 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ccbcf4-4dab-493d-9398-918eb0956ebd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:42.383606 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:42.388890 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 276/559] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:42.390035 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfbe45-a7b1-42c8-866a-f3d090f9bc31 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:42.477082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ccbcf4-4dab-493d-9398-918eb0956ebd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:42.478682 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ccbcf4-4dab-493d-9398-918eb0956ebd None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:42.497352 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:42.519939 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfbe45-a7b1-42c8-866a-f3d090f9bc31 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:42.574465 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4c94c-b54c-46e3-b845-aee53b551138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:42.575672 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4c94c-b54c-46e3-b845-aee53b551138 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:42.576021 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4c94c-b54c-46e3-b845-aee53b551138 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:42.576361 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4c94c-b54c-46e3-b845-aee53b551138 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:42.589702 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4c94c-b54c-46e3-b845-aee53b551138 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:42.591652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4c94c-b54c-46e3-b845-aee53b551138 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:42.618918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-b4f4c94c-b54c-46e3-b845-aee53b551138 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:42.618918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:42.618918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:42.618918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:42.618918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:42.618918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:42.618918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:42.618918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:42.618918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:42.618918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:42.618918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:42.618918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:42.618918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:42.618918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:42.618918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:42.620150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:42.620150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:42.620150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:42.620150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:42.620150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:42.620150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:42.620150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:42.620150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:42.620150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:42.620150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:42.620150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:42.620150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:42.620150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:42.620150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:42.620150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:42.620150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:42.620150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:42.621019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:42.621019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:42.621019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:42.621019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:42.621019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 277/560] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:42.621848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4c94c-b54c-46e3-b845-aee53b551138 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:42.622197 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 284/561] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:13:42.626465 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccbcf4-4dab-493d-9398-918eb0956ebd None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:42.680609 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:42.733851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4c94c-b54c-46e3-b845-aee53b551138 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:42.749321 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ccbcf4-4dab-493d-9398-918eb0956ebd None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:42.768906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98001169-d0ad-4de8-9712-6a434b916245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:42.769708 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98001169-d0ad-4de8-9712-6a434b916245 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:42.769990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98001169-d0ad-4de8-9712-6a434b916245 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:42.770268 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98001169-d0ad-4de8-9712-6a434b916245 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:42.777789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:42.781753 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98001169-d0ad-4de8-9712-6a434b916245 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:42.785249 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4525f3-8e07-415f-851b-c2b1656678eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:42.785912 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4525f3-8e07-415f-851b-c2b1656678eb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:42.786194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4525f3-8e07-415f-851b-c2b1656678eb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:42.786467 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4525f3-8e07-415f-851b-c2b1656678eb None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:42.787184 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4525f3-8e07-415f-851b-c2b1656678eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:42.787577 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4525f3-8e07-415f-851b-c2b1656678eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:42.789247 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4525f3-8e07-415f-851b-c2b1656678eb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:42.790047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-98001169-d0ad-4de8-9712-6a434b916245 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:42.790047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:42.790047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:42.790047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:42.790047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:42.790047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:42.790047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:42.790047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:42.790047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:42.790047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:42.790047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:42.790047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:42.790047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:42.790047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:42.790047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:42.790047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:42.794212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:42.794212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:42.794212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:42.794212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:42.794212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:42.794212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:42.794212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:42.794212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:42.794212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:42.794212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:42.794212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:42.794212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:42.794212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:42.794212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:42.794212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:42.794212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:42.795055 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:42.795055 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:42.795055 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:42.795055 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:42.795055 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:42.795055 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:42.795055 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:42.795055 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:42.795055 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:42.795055 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:42.795055 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:42.795055 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:42.795055 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 278/562] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 30 16:13:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:42.797084 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98001169-d0ad-4de8-9712-6a434b916245 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:42.801119 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 285/563] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:42.802349 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627a8a1-d915-4834-bddc-345c931074ab None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:42.802767 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627a8a1-d915-4834-bddc-345c931074ab None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:42.802939 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627a8a1-d915-4834-bddc-345c931074ab None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:42.805064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 286/564] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:42.807671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9650dc9-5503-4ac1-a3f6-0b4fddff8a7d None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:42.807969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9650dc9-5503-4ac1-a3f6-0b4fddff8a7d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:42.808252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9650dc9-5503-4ac1-a3f6-0b4fddff8a7d None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:42.809605 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 287/565] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:42.810347 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9650dc9-5503-4ac1-a3f6-0b4fddff8a7d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:42.909104 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98001169-d0ad-4de8-9712-6a434b916245 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:42.912388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:42.919833 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9650dc9-5503-4ac1-a3f6-0b4fddff8a7d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:42.944557 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776279cb-24a0-469f-adf6-1ae57cc71bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:42.945363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776279cb-24a0-469f-adf6-1ae57cc71bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:42.945652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776279cb-24a0-469f-adf6-1ae57cc71bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:42.945927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776279cb-24a0-469f-adf6-1ae57cc71bd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:42.954803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aede604-e63e-47c6-88f9-a08bca96580b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:42.955527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aede604-e63e-47c6-88f9-a08bca96580b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:42.955821 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aede604-e63e-47c6-88f9-a08bca96580b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:42.956098 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aede604-e63e-47c6-88f9-a08bca96580b None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:42.956807 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776279cb-24a0-469f-adf6-1ae57cc71bd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:42.956918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aede604-e63e-47c6-88f9-a08bca96580b None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:42.957137 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aede604-e63e-47c6-88f9-a08bca96580b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:42.958270 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776279cb-24a0-469f-adf6-1ae57cc71bd2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:42.958518 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aede604-e63e-47c6-88f9-a08bca96580b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:42.966595 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 288/566] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 30 16:13:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:42.967760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aede604-e63e-47c6-88f9-a08bca96580b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:42.980700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-776279cb-24a0-469f-adf6-1ae57cc71bd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:42.980700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:42.980700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:42.980700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:42.980700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:42.980700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:42.980700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:42.980700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:42.980700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:42.980700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:42.980700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:42.980700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:42.980700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:42.980700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:42.980700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:42.981764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:42.981764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:42.981764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:42.981764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:42.981764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:42.981764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:42.981764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:42.981764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:42.981764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:42.981764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:42.981764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:42.981764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:42.981764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:42.981764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:42.981764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:42.981764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:42.981764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:42.982716 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:42.982716 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:42.982716 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:42.982716 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:42.982716 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 279/567] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:42.983265 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f311431-385a-49f1-b291-184bb010515e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:42.983541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f311431-385a-49f1-b291-184bb010515e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:42.983791 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f311431-385a-49f1-b291-184bb010515e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:43.002571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7f311431-385a-49f1-b291-184bb010515e None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:43.003766 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7f311431-385a-49f1-b291-184bb010515e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:43.050922 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:43.071424 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aede604-e63e-47c6-88f9-a08bca96580b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:43.071914 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 280/568] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:43.073226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc356a3-2c92-4106-a40d-7a42c77e8757 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:43.073517 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc356a3-2c92-4106-a40d-7a42c77e8757 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:43.073799 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc356a3-2c92-4106-a40d-7a42c77e8757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:43.090422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5dc356a3-2c92-4106-a40d-7a42c77e8757 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:43.091655 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5dc356a3-2c92-4106-a40d-7a42c77e8757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:43.105871 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2a12a-8b94-48b6-9752-9b570bab7a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:43.106659 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2a12a-8b94-48b6-9752-9b570bab7a35 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:43.106940 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2a12a-8b94-48b6-9752-9b570bab7a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:43.107213 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2a12a-8b94-48b6-9752-9b570bab7a35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:43.113787 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2a12a-8b94-48b6-9752-9b570bab7a35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:43.115261 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2a12a-8b94-48b6-9752-9b570bab7a35 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:43.130315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-76a2a12a-8b94-48b6-9752-9b570bab7a35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:43.130315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:43.130315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:43.130315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:43.130315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:43.130315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:43.130315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:43.130315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:43.130315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:43.130315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:43.130315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:43.130315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:43.130315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:13:43.130315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:13:43.130315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:13:43.131320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:13:43.131320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:43.131320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:43.131320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:43.131320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:43.131320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:43.131320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:43.131320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:43.131320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:43.131320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:43.131320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:43.131320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:43.131320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:43.131320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:43.131320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:43.131320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:43.131320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:43.132260 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:13:43.132260 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:13:43.132260 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:13:43.132260 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:13:43.132260 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:13:43.132260 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:13:43.132260 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:43.132260 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:43.132260 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 289/569] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:13:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:43.133123 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2a12a-8b94-48b6-9752-9b570bab7a35 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:43.162339 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 281/570] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:43.163677 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc356a3-2c92-4106-a40d-7a42c77e8757 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:43.208501 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:43.240777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2a12a-8b94-48b6-9752-9b570bab7a35 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:43.265916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc356a3-2c92-4106-a40d-7a42c77e8757 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:43.277160 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd3d85-08ac-4d73-b170-e8a016ad884b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:43.277932 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd3d85-08ac-4d73-b170-e8a016ad884b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:43.278215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd3d85-08ac-4d73-b170-e8a016ad884b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:43.278493 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd3d85-08ac-4d73-b170-e8a016ad884b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:43.278953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:43.288944 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd3d85-08ac-4d73-b170-e8a016ad884b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:43.290408 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fd3d85-08ac-4d73-b170-e8a016ad884b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:43.301976 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5496b00-3c26-4ae9-86c9-b46a8e81bba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:43.302714 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5496b00-3c26-4ae9-86c9-b46a8e81bba4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:43.303115 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5496b00-3c26-4ae9-86c9-b46a8e81bba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:43.303953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5496b00-3c26-4ae9-86c9-b46a8e81bba4 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:43.304667 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5496b00-3c26-4ae9-86c9-b46a8e81bba4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:43.304972 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5496b00-3c26-4ae9-86c9-b46a8e81bba4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:43.306341 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5496b00-3c26-4ae9-86c9-b46a8e81bba4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:43.308916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f3fd3d85-08ac-4d73-b170-e8a016ad884b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:43.308916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:43.308916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:43.308916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:43.308916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:43.308916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:43.308916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:43.308916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:43.308916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:43.308916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:43.308916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:43.308916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:43.308916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:43.308916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:43.308916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:43.309790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:43.309790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:43.309790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:43.309790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:43.309790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:43.309790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:43.309790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:43.309790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:43.309790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:43.309790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:43.309790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:43.309790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:43.309790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:43.309790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:43.309790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:43.309790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:43.309790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:43.310664 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:43.310664 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:43.310664 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:43.310664 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:43.310664 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 290/571] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:43.311033 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd3d85-08ac-4d73-b170-e8a016ad884b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:43.317842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 282/572] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:43.318655 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5496b00-3c26-4ae9-86c9-b46a8e81bba4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:43.384963 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:43.419437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd3d85-08ac-4d73-b170-e8a016ad884b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:43.421934 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5496b00-3c26-4ae9-86c9-b46a8e81bba4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:43.457331 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594c74d-36c1-4e5b-9f8e-6a53c35235f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:43.457659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7d55a-6358-4cce-8842-3d3b71100d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:43.458122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594c74d-36c1-4e5b-9f8e-6a53c35235f2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:43.458315 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7d55a-6358-4cce-8842-3d3b71100d36 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:43.458440 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594c74d-36c1-4e5b-9f8e-6a53c35235f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:43.458600 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7d55a-6358-4cce-8842-3d3b71100d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:43.458708 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594c74d-36c1-4e5b-9f8e-6a53c35235f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:43.458875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7d55a-6358-4cce-8842-3d3b71100d36 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:43.459712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7d55a-6358-4cce-8842-3d3b71100d36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:43.460015 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7d55a-6358-4cce-8842-3d3b71100d36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:43.461488 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7d55a-6358-4cce-8842-3d3b71100d36 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:43.469510 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9594c74d-36c1-4e5b-9f8e-6a53c35235f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:43.473662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 283/573] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:13:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:43.474475 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7d55a-6358-4cce-8842-3d3b71100d36 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:43.479214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-9594c74d-36c1-4e5b-9f8e-6a53c35235f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:43.479214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:43.479214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:43.479214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:43.479214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:43.479214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:43.479214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:43.479214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:43.479214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:43.479214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:43.479214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:43.479214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:43.479214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:43.479214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:43.479214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:43.479214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:43.480216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:43.480216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:43.480216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:43.480216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:43.480216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:43.480216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:43.480216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:43.480216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:43.480216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:43.480216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:43.480216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:43.480216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:43.480216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:43.480216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:43.480216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:43.480216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:43.481049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:43.481049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:43.481049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:43.481049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:43.481049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:43.481049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:43.481049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:43.481049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:43.481049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:43.481049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:43.481049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:43.481049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:43.481049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 291/574] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 30 16:13:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:43.481753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594c74d-36c1-4e5b-9f8e-6a53c35235f2 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:43.543063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:43.611912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7d55a-6358-4cce-8842-3d3b71100d36 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:43.612476 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594c74d-36c1-4e5b-9f8e-6a53c35235f2 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:43.641689 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:43.646577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57e9c7-1bd1-49b3-bd63-30522264b389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:43.647609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57e9c7-1bd1-49b3-bd63-30522264b389 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:43.647891 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57e9c7-1bd1-49b3-bd63-30522264b389 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:43.648162 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57e9c7-1bd1-49b3-bd63-30522264b389 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:43.649955 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3692d-e35a-4537-92be-16391e5ab6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:43.650701 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3692d-e35a-4537-92be-16391e5ab6f0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:43.651023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3692d-e35a-4537-92be-16391e5ab6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:43.651318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3692d-e35a-4537-92be-16391e5ab6f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:43.652092 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3692d-e35a-4537-92be-16391e5ab6f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:43.656378 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3692d-e35a-4537-92be-16391e5ab6f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:43.656378 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3692d-e35a-4537-92be-16391e5ab6f0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:43.659349 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57e9c7-1bd1-49b3-bd63-30522264b389 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:43.666189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 292/575] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:43.666995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3692d-e35a-4537-92be-16391e5ab6f0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:43.667880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-da57e9c7-1bd1-49b3-bd63-30522264b389 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:43.667880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:43.667880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:43.667880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:43.667880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:43.667880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:43.667880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:43.667880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:43.667880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:43.667880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:43.667880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:43.667880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:43.667880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:43.667880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:43.667880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:43.667880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:43.668789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:43.668789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:43.668789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:43.668789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:43.668789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:43.668789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:43.668789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:43.668789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:43.668789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:43.668789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:43.668789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:43.668789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:43.668789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:43.668789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:43.668789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:43.668789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:43.669624 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:43.669624 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:43.669624 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:43.669624 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:43.669624 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:43.669624 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:43.669624 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:43.669624 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:43.669624 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:43.669624 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:43.669624 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:43.669624 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:43.669624 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 284/576] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 30 16:13:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:43.670318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57e9c7-1bd1-49b3-bd63-30522264b389 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:43.761810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:43.770608 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3692d-e35a-4537-92be-16391e5ab6f0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:43.771105 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57e9c7-1bd1-49b3-bd63-30522264b389 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:43.806220 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac33f96-c462-4614-a58d-686a527d4aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:43.806759 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a87b7-26f6-4d50-ad59-bab2d8968307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:43.806938 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac33f96-c462-4614-a58d-686a527d4aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:43.807224 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac33f96-c462-4614-a58d-686a527d4aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:43.807328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a87b7-26f6-4d50-ad59-bab2d8968307 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:43.807529 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac33f96-c462-4614-a58d-686a527d4aa1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:43.807625 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a87b7-26f6-4d50-ad59-bab2d8968307 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:43.807833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a87b7-26f6-4d50-ad59-bab2d8968307 None admin] PATH_INFO: `/v3/projects/19ef92ebcce94b90bf39ad3a58772a1d/users/71cca3708386459197abf6c67505bc5f/roles/57bc5df632b3407d97116ae34d92a5cb` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:43.814299 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac33f96-c462-4614-a58d-686a527d4aa1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:43.815777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac33f96-c462-4614-a58d-686a527d4aa1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:43.822614 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a87b7-26f6-4d50-ad59-bab2d8968307 None admin] RBAC: Authorizing `identity:create_grant(project_id=19ef92ebcce94b90bf39ad3a58772a1d, user_id=71cca3708386459197abf6c67505bc5f, role_id=57bc5df632b3407d97116ae34d92a5cb)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:43.824108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a87b7-26f6-4d50-ad59-bab2d8968307 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:43.829764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-bac33f96-c462-4614-a58d-686a527d4aa1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:43.829764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:43.829764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:43.829764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:43.829764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:43.829764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:43.829764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:43.829764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:43.829764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:43.829764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:43.829764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:43.829764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:43.829764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:13:43.829764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:13:43.829764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:13:43.830797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:13:43.830797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:43.830797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:43.830797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:43.830797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:43.830797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:43.830797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:43.830797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:43.830797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:43.830797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:43.830797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:43.830797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:43.830797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:43.830797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:43.830797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:43.830797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:43.830797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:43.831750 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:13:43.831750 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:13:43.831750 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:13:43.831750 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:13:43.831750 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:13:43.831750 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:13:43.831750 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:43.831750 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:43.831750 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 293/577] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:13:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:43.832251 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac33f96-c462-4614-a58d-686a527d4aa1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:43.882733 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:43.897964 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 285/578] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 30 16:13:43 2022] PUT /identity/v3/projects/19ef92ebcce94b90bf39ad3a58772a1d/users/71cca3708386459197abf6c67505bc5f/roles/57bc5df632b3407d97116ae34d92a5cb => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:13:43.900356 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a87b7-26f6-4d50-ad59-bab2d8968307 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:43.942033 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac33f96-c462-4614-a58d-686a527d4aa1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:43.985327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5dd35a-c17a-4080-9e7b-449a78709d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:43.986015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5dd35a-c17a-4080-9e7b-449a78709d33 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:43.986300 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5dd35a-c17a-4080-9e7b-449a78709d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:43.986583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5dd35a-c17a-4080-9e7b-449a78709d33 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:43.987351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5dd35a-c17a-4080-9e7b-449a78709d33 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:43.987761 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5dd35a-c17a-4080-9e7b-449a78709d33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:43.989226 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5dd35a-c17a-4080-9e7b-449a78709d33 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:44.003494 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 294/579] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 30 16:13:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:44.004701 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5dd35a-c17a-4080-9e7b-449a78709d33 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:44.039056 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a87b7-26f6-4d50-ad59-bab2d8968307 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:44.081356 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48e1fb-e815-4b3a-a111-17f37bfe158c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:44.082031 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48e1fb-e815-4b3a-a111-17f37bfe158c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:44.082332 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48e1fb-e815-4b3a-a111-17f37bfe158c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:44.082591 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48e1fb-e815-4b3a-a111-17f37bfe158c None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:44.083335 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e48e1fb-e815-4b3a-a111-17f37bfe158c None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:44.083672 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e48e1fb-e815-4b3a-a111-17f37bfe158c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:44.085834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e48e1fb-e815-4b3a-a111-17f37bfe158c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:44.094451 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 286/580] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 30 16:13:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:44.095102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48e1fb-e815-4b3a-a111-17f37bfe158c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:44.098433 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:44.123540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5dd35a-c17a-4080-9e7b-449a78709d33 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:44.137882 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:44.160827 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2aad7-6d00-46c8-8e7d-14346a921ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:44.161636 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2aad7-6d00-46c8-8e7d-14346a921ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:44.161925 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2aad7-6d00-46c8-8e7d-14346a921ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:44.162202 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2aad7-6d00-46c8-8e7d-14346a921ef6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:44.171522 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2aad7-6d00-46c8-8e7d-14346a921ef6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:44.173136 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2aad7-6d00-46c8-8e7d-14346a921ef6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:44.192172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-9bc2aad7-6d00-46c8-8e7d-14346a921ef6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 16:13:44.192172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:44.192172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:44.192172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:44.192172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:44.192172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:44.192172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:44.192172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:44.192172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:44.192172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:44.192172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:44.192172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:44.192172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:13:44.192172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:13:44.192172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:13:44.192172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:13:44.193275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:44.193275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:44.193275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:13:44.193275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:44.193275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:13:44.193275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:44.193275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:44.193275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:44.193275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:44.193275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:44.193275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:44.193275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:44.193275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:44.193275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:44.193275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:44.193275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:44.193275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:44.193275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:44.194252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:44.194252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:44.194252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:13:44.194252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:13:44.194252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:13:44.194252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:13:44.194252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:13:44.194252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:13:44.194252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 16:13:44.194252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:44.194252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 295/581] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:44.194878 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2aad7-6d00-46c8-8e7d-14346a921ef6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:44.213404 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48e1fb-e815-4b3a-a111-17f37bfe158c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:44.258310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beed055-0880-4db7-be6b-12cf42a3cd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:44.258973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beed055-0880-4db7-be6b-12cf42a3cd10 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:44.259253 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beed055-0880-4db7-be6b-12cf42a3cd10 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:44.259547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beed055-0880-4db7-be6b-12cf42a3cd10 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:44.260332 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beed055-0880-4db7-be6b-12cf42a3cd10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:44.260637 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beed055-0880-4db7-be6b-12cf42a3cd10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:44.262064 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beed055-0880-4db7-be6b-12cf42a3cd10 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:44.277539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 287/582] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:13:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:44.278366 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beed055-0880-4db7-be6b-12cf42a3cd10 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:44.309727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2aad7-6d00-46c8-8e7d-14346a921ef6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:44.345603 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724ceb1-5eb4-4324-b98f-4110aab4a24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:44.345797 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:44.346417 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724ceb1-5eb4-4324-b98f-4110aab4a24d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:44.346786 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724ceb1-5eb4-4324-b98f-4110aab4a24d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:44.347061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724ceb1-5eb4-4324-b98f-4110aab4a24d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:44.358370 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724ceb1-5eb4-4324-b98f-4110aab4a24d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:44.364654 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724ceb1-5eb4-4324-b98f-4110aab4a24d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:44.378275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-5724ceb1-5eb4-4324-b98f-4110aab4a24d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:44.378275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:44.378275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:44.378275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:44.378275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:44.378275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:44.378275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:44.378275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:44.378275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:44.378275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:44.378275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:44.378275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:44.378275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:44.378275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:44.378275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:44.379401 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:44.379401 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:44.379401 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:44.379401 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:44.379401 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:44.379401 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:44.379401 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:44.379401 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:44.379401 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:44.379401 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:44.379401 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:44.379401 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:44.379401 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:44.379401 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:44.379401 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:44.379401 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:44.379401 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:44.380354 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:44.380354 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:44.380354 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:44.380354 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:44.380354 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 296/583] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:44.380354 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724ceb1-5eb4-4324-b98f-4110aab4a24d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:44.392263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beed055-0880-4db7-be6b-12cf42a3cd10 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:44.411678 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:44.433644 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a2d88-4287-4bff-914b-b6d303823e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:44.434466 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a2d88-4287-4bff-914b-b6d303823e56 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:44.434749 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a2d88-4287-4bff-914b-b6d303823e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:44.435031 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a2d88-4287-4bff-914b-b6d303823e56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:44.445788 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a2d88-4287-4bff-914b-b6d303823e56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:44.447223 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a2d88-4287-4bff-914b-b6d303823e56 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:44.465382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-fa6a2d88-4287-4bff-914b-b6d303823e56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:44.465382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:44.465382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:44.465382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:44.465382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:44.465382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:44.465382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:44.465382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:44.465382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:44.465382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:44.465382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:44.465382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:44.465382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:44.465382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:44.465382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:44.466494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:44.466494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:44.466494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:44.466494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:44.466494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:44.466494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:44.466494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:44.466494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:44.466494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:44.466494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:44.466494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:44.466494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:44.466494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:44.466494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:44.466494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:44.466494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:44.466494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:44.467616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:44.467616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:44.467616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:44.467616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:44.471666 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 288/584] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:44.472519 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a2d88-4287-4bff-914b-b6d303823e56 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:44.497977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724ceb1-5eb4-4324-b98f-4110aab4a24d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:44.533933 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759aaf10-2be1-49bb-98e3-5493d8bbae07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:44.534755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759aaf10-2be1-49bb-98e3-5493d8bbae07 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:44.535192 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759aaf10-2be1-49bb-98e3-5493d8bbae07 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:44.535611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759aaf10-2be1-49bb-98e3-5493d8bbae07 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:44.536557 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759aaf10-2be1-49bb-98e3-5493d8bbae07 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:44.536971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759aaf10-2be1-49bb-98e3-5493d8bbae07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:44.539185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759aaf10-2be1-49bb-98e3-5493d8bbae07 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:44.556983 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 297/585] 10.222.0.67 () {56 vars in 1137 bytes} [Thu Jun 30 16:13:44 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:44.557868 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759aaf10-2be1-49bb-98e3-5493d8bbae07 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:44.588883 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a2d88-4287-4bff-914b-b6d303823e56 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:44.605439 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:44.626174 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e971c3c-6aae-42dd-9a2d-37579b9e2185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:44.626832 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e971c3c-6aae-42dd-9a2d-37579b9e2185 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:44.627116 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e971c3c-6aae-42dd-9a2d-37579b9e2185 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:44.627393 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e971c3c-6aae-42dd-9a2d-37579b9e2185 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:44.628466 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e971c3c-6aae-42dd-9a2d-37579b9e2185 None admin] RBAC: Adding query filter params (role.id=57bc5df632b3407d97116ae34d92a5cb, scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d, user.id=988b917eea4b48ffae32e23b8555a5c6) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:44.628780 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e971c3c-6aae-42dd-9a2d-37579b9e2185 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:44.630377 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e971c3c-6aae-42dd-9a2d-37579b9e2185 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:44.647669 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 289/586] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:13:44 2022] GET /identity/v3/role_assignments?user.id=988b917eea4b48ffae32e23b8555a5c6&scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d&role.id=57bc5df632b3407d97116ae34d92a5cb => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:44.647669 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e971c3c-6aae-42dd-9a2d-37579b9e2185 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:44.656508 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:44.676859 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759aaf10-2be1-49bb-98e3-5493d8bbae07 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:44.725666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc2c88-8037-4180-a5b8-16e1360e75f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:44.726472 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc2c88-8037-4180-a5b8-16e1360e75f5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:44.726751 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc2c88-8037-4180-a5b8-16e1360e75f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:44.727070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dc2c88-8037-4180-a5b8-16e1360e75f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:44.727869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc2c88-8037-4180-a5b8-16e1360e75f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:44.728182 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc2c88-8037-4180-a5b8-16e1360e75f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:44.729611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dc2c88-8037-4180-a5b8-16e1360e75f5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:44.748164 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 298/587] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:44.748164 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc2c88-8037-4180-a5b8-16e1360e75f5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:44.802966 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e971c3c-6aae-42dd-9a2d-37579b9e2185 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:44.845865 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31795d83-c260-444b-b179-0eadca450753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:44.846841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31795d83-c260-444b-b179-0eadca450753 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:44.847128 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31795d83-c260-444b-b179-0eadca450753 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:44.847407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31795d83-c260-444b-b179-0eadca450753 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:44.848423 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31795d83-c260-444b-b179-0eadca450753 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:44.848731 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31795d83-c260-444b-b179-0eadca450753 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:44.850291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31795d83-c260-444b-b179-0eadca450753 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:44.871702 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 290/588] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:44.872900 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31795d83-c260-444b-b179-0eadca450753 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:44.875006 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dc2c88-8037-4180-a5b8-16e1360e75f5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:44.911920 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b5c2a-0259-4654-870b-ff53df3a278b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:44.913474 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b5c2a-0259-4654-870b-ff53df3a278b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:44.914105 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b5c2a-0259-4654-870b-ff53df3a278b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:44.914105 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b5c2a-0259-4654-870b-ff53df3a278b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:44.931851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:44.934503 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b5c2a-0259-4654-870b-ff53df3a278b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:44.936699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b5c2a-0259-4654-870b-ff53df3a278b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:44.939288 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:44.956252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-805b5c2a-0259-4654-870b-ff53df3a278b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:44.956252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:44.956252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:44.956252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:44.956252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:44.956252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:44.956252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:44.956252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:44.956252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:44.956252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:44.956252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:44.956252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:44.956252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:44.956252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:44.956252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:44.957204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:44.957204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:44.957204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:44.957204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:44.957204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:44.957204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:44.957204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:44.957204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:44.957204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:44.957204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:44.957204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:44.957204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:44.957204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:44.957204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:44.957204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:44.957204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:44.957204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:44.958081 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:44.958081 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:44.958081 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:44.958081 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:44.960648 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 299/589] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:44.961743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b5c2a-0259-4654-870b-ff53df3a278b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:45.007904 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31795d83-c260-444b-b179-0eadca450753 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:45.048275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b546155-9684-464f-b33b-eb81ce9bf2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:45.049082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b546155-9684-464f-b33b-eb81ce9bf2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:45.049363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b546155-9684-464f-b33b-eb81ce9bf2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:45.049634 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b546155-9684-464f-b33b-eb81ce9bf2e8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:45.059154 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b546155-9684-464f-b33b-eb81ce9bf2e8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:45.061402 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b546155-9684-464f-b33b-eb81ce9bf2e8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:45.080790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b5c2a-0259-4654-870b-ff53df3a278b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:45.081438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-0b546155-9684-464f-b33b-eb81ce9bf2e8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 16:13:45.081438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:45.081438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:45.081438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:45.081438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:45.081438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:45.081438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:45.081438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:45.081438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:45.081438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:45.081438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:45.081438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:45.081438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:13:45.081438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:13:45.081438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:13:45.081438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:13:45.082437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:45.082437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:45.082437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:13:45.082437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:45.082437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:13:45.082437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:45.082437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:45.082437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:45.082437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:45.082437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:45.082437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:45.082437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:45.082437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:45.082437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:45.082437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:45.082437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:45.082437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:45.082437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:45.083366 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:45.083366 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:45.083366 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:13:45.083366 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:13:45.083366 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:13:45.083366 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:13:45.083366 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:13:45.083366 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:13:45.083366 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 16:13:45.083366 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:45.083945 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 291/590] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:45.085041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b546155-9684-464f-b33b-eb81ce9bf2e8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:45.120689 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46594e13-4290-49e0-b1ce-aeabd30f6f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:45.121807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46594e13-4290-49e0-b1ce-aeabd30f6f56 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:45.122040 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46594e13-4290-49e0-b1ce-aeabd30f6f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:45.122746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46594e13-4290-49e0-b1ce-aeabd30f6f56 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:45.133883 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46594e13-4290-49e0-b1ce-aeabd30f6f56 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:45.135357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46594e13-4290-49e0-b1ce-aeabd30f6f56 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:45.155714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-46594e13-4290-49e0-b1ce-aeabd30f6f56 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 16:13:45.155714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:45.155714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:45.155714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:45.155714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:45.155714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:45.155714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:45.155714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:45.155714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:45.155714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:45.155714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:45.155714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:45.155714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:13:45.155714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:13:45.155714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:13:45.155714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:13:45.156691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:45.156691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:45.156691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:13:45.156691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:45.156691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:13:45.156691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:45.156691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:45.156691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:45.156691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:45.156691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:45.156691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:45.156691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:45.156691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:45.156691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:45.156691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:45.156691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:45.156691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:45.156691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:45.157607 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:45.157607 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:45.157607 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:13:45.157607 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:13:45.157607 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:13:45.157607 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:13:45.157607 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:13:45.157607 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:13:45.157607 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 16:13:45.157607 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:45.158137 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 300/591] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 30 16:13:44 2022] GET /identity/v3/users/neutron => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:45.162639 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46594e13-4290-49e0-b1ce-aeabd30f6f56 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:45.208355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:45.212908 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b546155-9684-464f-b33b-eb81ce9bf2e8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:45.232932 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:45.255288 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8647c-3366-4c45-9120-571cfdb39610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:45.256875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8647c-3366-4c45-9120-571cfdb39610 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:45.257312 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8647c-3366-4c45-9120-571cfdb39610 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:45.262498 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e8647c-3366-4c45-9120-571cfdb39610 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:45.262498 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8647c-3366-4c45-9120-571cfdb39610 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:45.262498 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8647c-3366-4c45-9120-571cfdb39610 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:45.262498 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e8647c-3366-4c45-9120-571cfdb39610 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:45.277783 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 292/592] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:45.278667 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8647c-3366-4c45-9120-571cfdb39610 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:45.280654 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46594e13-4290-49e0-b1ce-aeabd30f6f56 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:45.315642 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41be992-ac25-41b8-bc9b-4668aabab870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:45.316321 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41be992-ac25-41b8-bc9b-4668aabab870 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:45.316604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41be992-ac25-41b8-bc9b-4668aabab870 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:45.316879 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41be992-ac25-41b8-bc9b-4668aabab870 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:45.317724 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41be992-ac25-41b8-bc9b-4668aabab870 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:45.318035 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41be992-ac25-41b8-bc9b-4668aabab870 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:45.320368 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41be992-ac25-41b8-bc9b-4668aabab870 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:45.339050 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 301/593] 10.222.0.67 () {56 vars in 1137 bytes} [Thu Jun 30 16:13:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:45.340008 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41be992-ac25-41b8-bc9b-4668aabab870 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:45.391462 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e8647c-3366-4c45-9120-571cfdb39610 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:45.430479 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c451e3-9067-4919-9776-aaab7f845cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:45.431164 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c451e3-9067-4919-9776-aaab7f845cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:45.431450 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c451e3-9067-4919-9776-aaab7f845cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:45.431778 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c451e3-9067-4919-9776-aaab7f845cf6 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:45.432783 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c451e3-9067-4919-9776-aaab7f845cf6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:45.433103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c451e3-9067-4919-9776-aaab7f845cf6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:45.434711 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c451e3-9067-4919-9776-aaab7f845cf6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:45.442292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41be992-ac25-41b8-bc9b-4668aabab870 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:45.455184 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 293/594] 10.222.0.67 () {56 vars in 1139 bytes} [Thu Jun 30 16:13:45 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:45.456217 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c451e3-9067-4919-9776-aaab7f845cf6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:45.457140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:45.477022 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:45.479725 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292230b-08c0-475d-adb4-4f480af86bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:45.480528 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292230b-08c0-475d-adb4-4f480af86bfe None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:45.480812 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292230b-08c0-475d-adb4-4f480af86bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:45.481090 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292230b-08c0-475d-adb4-4f480af86bfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:45.488542 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292230b-08c0-475d-adb4-4f480af86bfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:45.490061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292230b-08c0-475d-adb4-4f480af86bfe None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:45.504204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-e292230b-08c0-475d-adb4-4f480af86bfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:45.504204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:45.504204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:45.504204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:45.504204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:45.504204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:45.504204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:45.504204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:45.504204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:45.504204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:45.504204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:45.504204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:45.504204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:13:45.504204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:13:45.504204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:13:45.505327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:13:45.505327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:45.505327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:45.505327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:45.505327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:45.505327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:45.505327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:45.505327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:45.505327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:45.505327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:45.505327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:45.505327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:45.505327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:45.505327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:45.505327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:45.505327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:45.505327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:45.506914 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:13:45.506914 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:13:45.506914 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:13:45.506914 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:13:45.506914 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:13:45.506914 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:13:45.506914 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:45.506914 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:45.510825 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 302/595] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:13:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:45.511699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292230b-08c0-475d-adb4-4f480af86bfe None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:45.566742 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c451e3-9067-4919-9776-aaab7f845cf6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:45.605271 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dcf3e8-4fee-4f7a-99df-a7d9b7be3a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:45.606452 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dcf3e8-4fee-4f7a-99df-a7d9b7be3a6e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:45.606886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dcf3e8-4fee-4f7a-99df-a7d9b7be3a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:45.607335 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dcf3e8-4fee-4f7a-99df-a7d9b7be3a6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:45.619103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dcf3e8-4fee-4f7a-99df-a7d9b7be3a6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:45.620898 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dcf3e8-4fee-4f7a-99df-a7d9b7be3a6e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:45.629551 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292230b-08c0-475d-adb4-4f480af86bfe None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:45.640937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-c9dcf3e8-4fee-4f7a-99df-a7d9b7be3a6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:45.640937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:45.640937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:45.640937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:45.640937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:45.640937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:45.640937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:45.640937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:45.640937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:45.640937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:45.640937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:45.640937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:45.640937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:45.640937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:45.640937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:45.641931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:45.641931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:45.641931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:45.641931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:45.641931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:45.641931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:45.641931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:45.641931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:45.641931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:45.641931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:45.641931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:45.641931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:45.641931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:45.641931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:45.641931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:45.641931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:45.641931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:45.642826 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:45.642826 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:45.642826 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:45.642826 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:45.644899 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 294/596] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:45.645997 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dcf3e8-4fee-4f7a-99df-a7d9b7be3a6e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:45.669666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a2d8e-74a3-4851-8cb2-f22a2a281edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:45.670464 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a2d8e-74a3-4851-8cb2-f22a2a281edc None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:45.670744 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a2d8e-74a3-4851-8cb2-f22a2a281edc None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:45.671016 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a2d8e-74a3-4851-8cb2-f22a2a281edc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:45.686849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a2d8e-74a3-4851-8cb2-f22a2a281edc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:45.688519 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a2d8e-74a3-4851-8cb2-f22a2a281edc None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:45.710763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-af7a2d8e-74a3-4851-8cb2-f22a2a281edc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:45.710763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:45.710763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:45.710763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:45.710763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:45.710763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:45.710763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:45.710763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:45.710763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:45.710763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:45.710763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:45.710763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:45.710763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:45.710763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:45.710763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:45.712082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:45.712082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:45.712082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:45.712082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:45.712082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:45.712082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:45.712082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:45.712082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:45.712082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:45.712082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:45.712082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:45.712082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:45.712082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:45.712082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:45.712082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:45.712082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:45.712082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:45.712950 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:45.712950 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:45.712950 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:45.712950 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:45.712950 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 303/597] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:45.713233 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a2d8e-74a3-4851-8cb2-f22a2a281edc None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:45.727275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:45.743760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:45.775813 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dcf3e8-4fee-4f7a-99df-a7d9b7be3a6e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:45.814304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd454a-a591-4ef4-b991-1c64b3623de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:45.815018 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd454a-a591-4ef4-b991-1c64b3623de0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:45.815324 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd454a-a591-4ef4-b991-1c64b3623de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:45.815595 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd454a-a591-4ef4-b991-1c64b3623de0 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:45.816389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd454a-a591-4ef4-b991-1c64b3623de0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:45.816777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd454a-a591-4ef4-b991-1c64b3623de0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:45.818163 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcd454a-a591-4ef4-b991-1c64b3623de0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:45.830892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 295/598] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:13:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:45.831687 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd454a-a591-4ef4-b991-1c64b3623de0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:45.843066 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a2d8e-74a3-4851-8cb2-f22a2a281edc None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:45.884664 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab6ca5-6882-4aa2-961e-9177e2808082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:45.886354 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab6ca5-6882-4aa2-961e-9177e2808082 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:45.886864 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab6ca5-6882-4aa2-961e-9177e2808082 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:45.887443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab6ca5-6882-4aa2-961e-9177e2808082 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:45.888466 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab6ca5-6882-4aa2-961e-9177e2808082 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:45.889002 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab6ca5-6882-4aa2-961e-9177e2808082 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:45.890739 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab6ca5-6882-4aa2-961e-9177e2808082 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:45.907049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 304/599] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:45.908333 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab6ca5-6882-4aa2-961e-9177e2808082 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:45.945198 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd454a-a591-4ef4-b991-1c64b3623de0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:45.972806 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:45.980747 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805c6e4-6ee9-437b-82aa-b476d69a791f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:45.981461 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805c6e4-6ee9-437b-82aa-b476d69a791f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:45.981786 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805c6e4-6ee9-437b-82aa-b476d69a791f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:45.982056 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805c6e4-6ee9-437b-82aa-b476d69a791f None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:45.982780 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805c6e4-6ee9-437b-82aa-b476d69a791f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:45.994784 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805c6e4-6ee9-437b-82aa-b476d69a791f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:45.994784 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805c6e4-6ee9-437b-82aa-b476d69a791f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:46.001392 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 296/600] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:46.002328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805c6e4-6ee9-437b-82aa-b476d69a791f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:46.022728 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:46.040004 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab6ca5-6882-4aa2-961e-9177e2808082 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:46.075980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6011861-777c-431f-965c-f16bd87654f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:46.077041 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6011861-777c-431f-965c-f16bd87654f7 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:46.077329 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6011861-777c-431f-965c-f16bd87654f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:46.077668 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6011861-777c-431f-965c-f16bd87654f7 None admin] PATH_INFO: `/v3/projects/19ef92ebcce94b90bf39ad3a58772a1d/users/61437a2fa2a94b35b1b4a5750f6e2cad/roles/57bc5df632b3407d97116ae34d92a5cb` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:46.096181 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6011861-777c-431f-965c-f16bd87654f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=19ef92ebcce94b90bf39ad3a58772a1d, user_id=61437a2fa2a94b35b1b4a5750f6e2cad, role_id=57bc5df632b3407d97116ae34d92a5cb)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:46.097952 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6011861-777c-431f-965c-f16bd87654f7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:46.118306 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805c6e4-6ee9-437b-82aa-b476d69a791f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:46.154743 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ea6a5-a66d-4211-85a5-3b67362473a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:46.155555 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ea6a5-a66d-4211-85a5-3b67362473a6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:46.155837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ea6a5-a66d-4211-85a5-3b67362473a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:46.156123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ea6a5-a66d-4211-85a5-3b67362473a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:46.163137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ea6a5-a66d-4211-85a5-3b67362473a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:46.164541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ea6a5-a66d-4211-85a5-3b67362473a6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:46.177956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e30ea6a5-a66d-4211-85a5-3b67362473a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:46.177956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:46.177956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:46.177956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:46.177956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:46.177956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:46.177956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:46.177956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:46.177956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:46.177956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:46.177956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:46.177956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:46.177956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:13:46.177956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:13:46.177956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:13:46.187057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:13:46.187057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:46.187057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:46.187057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:46.187057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:46.187057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:46.187057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:46.187057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:46.187057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:46.187057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:46.187057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:46.187057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:46.187057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:46.187057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:46.187057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:46.187057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:46.187057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:46.188278 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:13:46.188278 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:13:46.188278 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:13:46.188278 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:13:46.188278 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:13:46.188278 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:13:46.188278 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:46.188278 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:46.188793 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 305/601] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 30 16:13:45 2022] PUT /identity/v3/projects/19ef92ebcce94b90bf39ad3a58772a1d/users/61437a2fa2a94b35b1b4a5750f6e2cad/roles/57bc5df632b3407d97116ae34d92a5cb => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:13:46.188793 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6011861-777c-431f-965c-f16bd87654f7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:46.188945 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 297/602] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:13:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:46.188945 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc5675-7bdc-485e-8401-6fe01f3b7f11 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:46.188945 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc5675-7bdc-485e-8401-6fe01f3b7f11 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:46.188945 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc5675-7bdc-485e-8401-6fe01f3b7f11 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:46.188945 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 298/603] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:46.188945 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc5675-7bdc-485e-8401-6fe01f3b7f11 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:46.252313 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:46.329201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6011861-777c-431f-965c-f16bd87654f7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:46.335999 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc5675-7bdc-485e-8401-6fe01f3b7f11 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:46.378209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d9fad-bda8-434e-b74a-23d8053a8398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:46.379142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d9fad-bda8-434e-b74a-23d8053a8398 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:46.379530 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d9fad-bda8-434e-b74a-23d8053a8398 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:46.379820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d9fad-bda8-434e-b74a-23d8053a8398 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:46.380811 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2fb44e-86e1-4f30-a392-9facc46733bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:46.381803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fb44e-86e1-4f30-a392-9facc46733bc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:46.382519 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:46.382738 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fb44e-86e1-4f30-a392-9facc46733bc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:46.382875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2fb44e-86e1-4f30-a392-9facc46733bc None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:46.384511 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2fb44e-86e1-4f30-a392-9facc46733bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:46.384816 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2fb44e-86e1-4f30-a392-9facc46733bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:46.388890 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2fb44e-86e1-4f30-a392-9facc46733bc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:46.392762 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d9fad-bda8-434e-b74a-23d8053a8398 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:46.394526 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d9fad-bda8-434e-b74a-23d8053a8398 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:46.405628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 299/604] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:13:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:46.411499 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafbd3e-8bf7-418d-9928-58dd7b95d0a0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:46.411499 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafbd3e-8bf7-418d-9928-58dd7b95d0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:46.411499 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dafbd3e-8bf7-418d-9928-58dd7b95d0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:46.415671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-331d9fad-bda8-434e-b74a-23d8053a8398 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:46.415671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:46.415671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:46.415671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:46.415671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:46.415671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:46.415671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:46.415671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:46.415671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:46.415671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:46.415671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:46.415671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:46.415671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:13:46.415671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:13:46.415671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:13:46.416646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:13:46.416646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:46.416646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:46.416646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:46.416646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:46.416646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:46.416646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:46.416646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:46.416646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:46.416646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:46.416646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:46.416646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:46.416646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:46.416646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:46.416646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:46.416646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:46.416646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:46.417503 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:13:46.417503 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:13:46.417503 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:13:46.417503 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:13:46.417503 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:13:46.417503 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:13:46.417503 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:46.417503 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:46.417503 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 306/605] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:13:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:46.418000 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d9fad-bda8-434e-b74a-23d8053a8398 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:46.433823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9dafbd3e-8bf7-418d-9928-58dd7b95d0a0 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:46.435291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9dafbd3e-8bf7-418d-9928-58dd7b95d0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:46.526355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 300/606] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:46.528065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dafbd3e-8bf7-418d-9928-58dd7b95d0a0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:46.550122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d9fad-bda8-434e-b74a-23d8053a8398 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:46.587209 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:46.588756 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85e25a-8ed3-4cea-b26f-1df17c14b993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:46.589673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85e25a-8ed3-4cea-b26f-1df17c14b993 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:46.589956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85e25a-8ed3-4cea-b26f-1df17c14b993 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:46.590236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85e25a-8ed3-4cea-b26f-1df17c14b993 None admin] PATH_INFO: `/v3/projects/19ef92ebcce94b90bf39ad3a58772a1d/users/ec41c84c9375437590c42621ca5b400a/roles/57bc5df632b3407d97116ae34d92a5cb` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:46.606474 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb85e25a-8ed3-4cea-b26f-1df17c14b993 None admin] RBAC: Authorizing `identity:create_grant(project_id=19ef92ebcce94b90bf39ad3a58772a1d, user_id=ec41c84c9375437590c42621ca5b400a, role_id=57bc5df632b3407d97116ae34d92a5cb)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:46.607975 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb85e25a-8ed3-4cea-b26f-1df17c14b993 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:46.659072 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dafbd3e-8bf7-418d-9928-58dd7b95d0a0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:46.698061 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47ef74-6b85-4087-9c83-a5bccb3b70f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:46.698997 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47ef74-6b85-4087-9c83-a5bccb3b70f9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:46.699445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47ef74-6b85-4087-9c83-a5bccb3b70f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:46.699890 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47ef74-6b85-4087-9c83-a5bccb3b70f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:46.700809 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e47ef74-6b85-4087-9c83-a5bccb3b70f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:46.701259 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e47ef74-6b85-4087-9c83-a5bccb3b70f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:46.702890 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e47ef74-6b85-4087-9c83-a5bccb3b70f9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:46.720619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 301/607] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:13:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:46.720619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47ef74-6b85-4087-9c83-a5bccb3b70f9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:46.735154 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 307/608] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 30 16:13:46 2022] PUT /identity/v3/projects/19ef92ebcce94b90bf39ad3a58772a1d/users/ec41c84c9375437590c42621ca5b400a/roles/57bc5df632b3407d97116ae34d92a5cb => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:13:46.740283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85e25a-8ed3-4cea-b26f-1df17c14b993 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:46.837997 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:46.842417 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:46.847389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47ef74-6b85-4087-9c83-a5bccb3b70f9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:46.889747 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8908a9-4f38-4d70-8c3f-5d3c27635756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:46.890941 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8908a9-4f38-4d70-8c3f-5d3c27635756 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:46.891376 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8908a9-4f38-4d70-8c3f-5d3c27635756 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:46.891843 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8908a9-4f38-4d70-8c3f-5d3c27635756 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:46.902170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85e25a-8ed3-4cea-b26f-1df17c14b993 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:46.905623 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8908a9-4f38-4d70-8c3f-5d3c27635756 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:46.916023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-5c8908a9-4f38-4d70-8c3f-5d3c27635756 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:46.916023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:46.916023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:46.916023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:46.916023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:46.916023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:46.916023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:46.916023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:46.916023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:46.916023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:46.916023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:46.916023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:46.916023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:46.916023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:46.916023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:46.916023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:46.923236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:46.923236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:46.923236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:46.923236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:46.923236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:46.923236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:46.923236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:46.923236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:46.923236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:46.923236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:46.923236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:46.923236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:46.923236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:46.923236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:46.923236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:46.923236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:46.924249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:46.924249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:46.924249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:46.924249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:46.924249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:46.924249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:46.924249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:46.924249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:46.924249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:46.924249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:46.924249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:46.924249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:46.924249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 302/609] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 30 16:13:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:46.924249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b9c19-aa89-4418-a783-146be23aa3f6 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:46.925211 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b9c19-aa89-4418-a783-146be23aa3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:46.925211 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b9c19-aa89-4418-a783-146be23aa3f6 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:46.925211 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 303/610] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:46.930119 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b9c19-aa89-4418-a783-146be23aa3f6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:46.948237 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bae227-71ac-4877-b9b7-db243b8d3a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:46.949781 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bae227-71ac-4877-b9b7-db243b8d3a26 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:46.950025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bae227-71ac-4877-b9b7-db243b8d3a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:46.950557 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bae227-71ac-4877-b9b7-db243b8d3a26 None admin] PATH_INFO: `/v3/projects/19ef92ebcce94b90bf39ad3a58772a1d/users/9b508dd8b7244d5f979e1ce7787d2c67/roles/57bc5df632b3407d97116ae34d92a5cb` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:46.973905 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bae227-71ac-4877-b9b7-db243b8d3a26 None admin] RBAC: Authorizing `identity:create_grant(project_id=19ef92ebcce94b90bf39ad3a58772a1d, user_id=9b508dd8b7244d5f979e1ce7787d2c67, role_id=57bc5df632b3407d97116ae34d92a5cb)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:46.976406 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bae227-71ac-4877-b9b7-db243b8d3a26 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:47.056117 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 308/611] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 30 16:13:46 2022] PUT /identity/v3/projects/19ef92ebcce94b90bf39ad3a58772a1d/users/9b508dd8b7244d5f979e1ce7787d2c67/roles/57bc5df632b3407d97116ae34d92a5cb => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:13:47.058706 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede3596-ada0-4244-85c5-38bb41d20375 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:47.059287 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede3596-ada0-4244-85c5-38bb41d20375 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:47.060061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede3596-ada0-4244-85c5-38bb41d20375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:47.069662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b9c19-aa89-4418-a783-146be23aa3f6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:47.083180 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-4ede3596-ada0-4244-85c5-38bb41d20375 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:47.085681 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4ede3596-ada0-4244-85c5-38bb41d20375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:47.142926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab6fd5-77de-4d50-893a-e37430e74fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:47.143421 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab6fd5-77de-4d50-893a-e37430e74fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:47.143728 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab6fd5-77de-4d50-893a-e37430e74fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:47.144001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab6fd5-77de-4d50-893a-e37430e74fb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:47.144806 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab6fd5-77de-4d50-893a-e37430e74fb5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:47.145050 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab6fd5-77de-4d50-893a-e37430e74fb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:47.146618 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab6fd5-77de-4d50-893a-e37430e74fb5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:47.168738 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 304/612] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 30 16:13:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:47.176378 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab6fd5-77de-4d50-893a-e37430e74fb5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:47.186494 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 309/613] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:47.225130 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede3596-ada0-4244-85c5-38bb41d20375 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:47.234916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:47.254569 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:47.347440 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab6fd5-77de-4d50-893a-e37430e74fb5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:47.354491 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede3596-ada0-4244-85c5-38bb41d20375 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:47.385649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815fb3d-3ae9-4706-8170-9f7c18986a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:47.386825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815fb3d-3ae9-4706-8170-9f7c18986a31 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:47.387304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815fb3d-3ae9-4706-8170-9f7c18986a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:47.387778 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4815fb3d-3ae9-4706-8170-9f7c18986a31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:47.399776 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815fb3d-3ae9-4706-8170-9f7c18986a31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:47.401541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4815fb3d-3ae9-4706-8170-9f7c18986a31 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:47.401649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4b866-c949-4cee-b6ef-a7d89922d7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:47.402516 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4b866-c949-4cee-b6ef-a7d89922d7d4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:47.402871 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4b866-c949-4cee-b6ef-a7d89922d7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:47.403213 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4b866-c949-4cee-b6ef-a7d89922d7d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:47.415909 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f4b866-c949-4cee-b6ef-a7d89922d7d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:47.422739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-4815fb3d-3ae9-4706-8170-9f7c18986a31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:47.422739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:47.422739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:47.422739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:47.422739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:47.422739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:47.422739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:47.422739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:47.422739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:47.422739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:47.422739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:47.422739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:47.422739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:47.422739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:47.422739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:47.423649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:47.423649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:47.423649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:47.423649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:47.423649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:47.423649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:47.423649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:47.423649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:47.423649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:47.423649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:47.423649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:47.423649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:47.423649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:47.423649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:47.423649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:47.423649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:47.423649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:47.424539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:47.424539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:47.424539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:47.424539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:47.424757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-d3f4b866-c949-4cee-b6ef-a7d89922d7d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:47.424757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:47.424757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:47.424757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:47.424757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:47.424757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:47.424757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:47.424757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:47.424757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:47.424757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:47.424757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:47.424757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:47.424757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:47.424757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:47.424757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:47.424757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:47.425586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:47.425586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:47.425586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:47.425586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:47.425586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:47.425586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:47.425586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:47.425586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:47.425586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:47.425586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:47.425586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:47.425586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:47.425586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:47.425586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:47.425586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:47.425586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:47.426543 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:47.426543 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:47.426543 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:47.426543 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:47.426543 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:47.426543 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:47.426543 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:47.426543 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:47.426543 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:47.426543 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:47.426543 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:47.426543 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:47.426543 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 310/614] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 30 16:13:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:47.433906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 305/615] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:47.435108 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4b866-c949-4cee-b6ef-a7d89922d7d4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:47.435490 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815fb3d-3ae9-4706-8170-9f7c18986a31 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:47.513190 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:47.543902 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4815fb3d-3ae9-4706-8170-9f7c18986a31 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:47.587288 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eedb675-e80a-4f17-9b8e-68aea6f696b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:47.589621 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedb675-e80a-4f17-9b8e-68aea6f696b8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:47.590050 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedb675-e80a-4f17-9b8e-68aea6f696b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:47.590478 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedb675-e80a-4f17-9b8e-68aea6f696b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:47.591334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eedb675-e80a-4f17-9b8e-68aea6f696b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:47.593495 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eedb675-e80a-4f17-9b8e-68aea6f696b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:47.595223 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eedb675-e80a-4f17-9b8e-68aea6f696b8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:47.617250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 306/616] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:47.623334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eedb675-e80a-4f17-9b8e-68aea6f696b8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:47.624212 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:47.658622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4b866-c949-4cee-b6ef-a7d89922d7d4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:47.712045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d0fb4-e68b-47fb-a41d-0c44812d5081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:47.712956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d0fb4-e68b-47fb-a41d-0c44812d5081 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:47.713443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d0fb4-e68b-47fb-a41d-0c44812d5081 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:47.713889 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9d0fb4-e68b-47fb-a41d-0c44812d5081 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:47.714832 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9d0fb4-e68b-47fb-a41d-0c44812d5081 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:47.715299 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9d0fb4-e68b-47fb-a41d-0c44812d5081 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:47.716849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9d0fb4-e68b-47fb-a41d-0c44812d5081 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:47.735673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 311/617] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 30 16:13:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:47.741748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d0fb4-e68b-47fb-a41d-0c44812d5081 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:47.821119 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eedb675-e80a-4f17-9b8e-68aea6f696b8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:47.857912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:47.864397 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5ba8a-f32d-4b0c-a4e2-eb150c6173a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:47.872245 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5ba8a-f32d-4b0c-a4e2-eb150c6173a9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:47.872245 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5ba8a-f32d-4b0c-a4e2-eb150c6173a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:47.872245 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5ba8a-f32d-4b0c-a4e2-eb150c6173a9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:47.876411 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5ba8a-f32d-4b0c-a4e2-eb150c6173a9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:47.878448 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5ba8a-f32d-4b0c-a4e2-eb150c6173a9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:47.897355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-73f5ba8a-f32d-4b0c-a4e2-eb150c6173a9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 16:13:47.897355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:47.897355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:47.897355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:47.897355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:47.897355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:47.897355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:47.897355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:47.897355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:47.897355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:47.897355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:47.897355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:47.897355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:13:47.897355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:13:47.897355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:13:47.897355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:13:47.898656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:47.898656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:47.898656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:13:47.898656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:47.898656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:13:47.898656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:47.898656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:47.898656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:47.898656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:47.898656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:47.898656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:47.898656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:47.898656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:47.898656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:47.898656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:47.898656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:47.898656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:47.898656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:47.900205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:47.900205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:47.900205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:13:47.900205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:13:47.900205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:13:47.900205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:13:47.900205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:13:47.900205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:13:47.900205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 16:13:47.900205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:47.900205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 307/618] 10.222.0.67 () {56 vars in 1093 bytes} [Thu Jun 30 16:13:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:47.904799 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5ba8a-f32d-4b0c-a4e2-eb150c6173a9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:47.944674 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9d0fb4-e68b-47fb-a41d-0c44812d5081 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:47.983077 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e1ae6-611d-412f-810b-df394cfa406c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:47.983904 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e1ae6-611d-412f-810b-df394cfa406c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:47.984191 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e1ae6-611d-412f-810b-df394cfa406c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:47.984465 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e1ae6-611d-412f-810b-df394cfa406c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:47.997092 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e1ae6-611d-412f-810b-df394cfa406c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:47.998586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e1ae6-611d-412f-810b-df394cfa406c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:48.014590 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:48.037534 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5ba8a-f32d-4b0c-a4e2-eb150c6173a9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:48.040221 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ad5e1ae6-611d-412f-810b-df394cfa406c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:48.040221 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:48.040221 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:48.040221 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:48.040221 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:48.040221 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:48.040221 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:48.040221 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:48.040221 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:48.040221 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:48.040221 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:48.040221 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:48.040221 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:48.040221 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:48.040221 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:48.041082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:48.041082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:48.041082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:48.041082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:48.041082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:48.041082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:48.041082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:48.041082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:48.041082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:48.041082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:48.041082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:48.041082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:48.041082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:48.041082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:48.041082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:48.041082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:48.041082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:48.042370 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:48.042370 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:48.042370 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:48.042370 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:48.042370 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 312/619] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:48.048675 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e1ae6-611d-412f-810b-df394cfa406c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:48.089944 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086c702-6788-4ebd-8721-a867248d3aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:48.090545 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086c702-6788-4ebd-8721-a867248d3aea None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:48.090827 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086c702-6788-4ebd-8721-a867248d3aea None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:48.091101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086c702-6788-4ebd-8721-a867248d3aea None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:48.092025 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086c702-6788-4ebd-8721-a867248d3aea None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:48.092362 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086c702-6788-4ebd-8721-a867248d3aea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:48.093873 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6086c702-6788-4ebd-8721-a867248d3aea None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:48.132029 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 308/620] 10.222.0.67 () {56 vars in 1133 bytes} [Thu Jun 30 16:13:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:48.137383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086c702-6788-4ebd-8721-a867248d3aea None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:48.213656 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e1ae6-611d-412f-810b-df394cfa406c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:48.215615 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:48.266075 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6086c702-6788-4ebd-8721-a867248d3aea None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:48.304693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8956f-994b-4d0f-9333-2f920528bb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:48.305374 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8956f-994b-4d0f-9333-2f920528bb2f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:48.305657 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8956f-994b-4d0f-9333-2f920528bb2f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:48.305935 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8956f-994b-4d0f-9333-2f920528bb2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:48.312735 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6f4c3-2c74-4bf6-a789-af935b4b22de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:48.313418 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6f4c3-2c74-4bf6-a789-af935b4b22de None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:48.313707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6f4c3-2c74-4bf6-a789-af935b4b22de None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:48.313990 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a6f4c3-2c74-4bf6-a789-af935b4b22de None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:48.314721 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6f4c3-2c74-4bf6-a789-af935b4b22de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:48.315033 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6f4c3-2c74-4bf6-a789-af935b4b22de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:48.316453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a6f4c3-2c74-4bf6-a789-af935b4b22de None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:48.345851 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 313/621] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:48.351417 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6f4c3-2c74-4bf6-a789-af935b4b22de None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:48.379162 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8956f-994b-4d0f-9333-2f920528bb2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:48.380725 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8956f-994b-4d0f-9333-2f920528bb2f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:48.461969 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:48.493492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:48.558671 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 309/622] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:13:48.563792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8956f-994b-4d0f-9333-2f920528bb2f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:48.580621 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a6f4c3-2c74-4bf6-a789-af935b4b22de None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:48.640996 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd49d1-5675-422c-a0e5-6823f8a58548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:48.642015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd49d1-5675-422c-a0e5-6823f8a58548 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:48.642444 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd49d1-5675-422c-a0e5-6823f8a58548 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:48.642860 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd49d1-5675-422c-a0e5-6823f8a58548 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:48.651704 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd49d1-5675-422c-a0e5-6823f8a58548 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:48.653321 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd49d1-5675-422c-a0e5-6823f8a58548 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:48.669046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-c5cd49d1-5675-422c-a0e5-6823f8a58548 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 16:13:48.669046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:48.669046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:48.669046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:48.669046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:48.669046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:48.669046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:48.669046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:48.669046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:48.669046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:48.669046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:48.669046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:48.669046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:13:48.669046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:13:48.669046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:13:48.669046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:13:48.670194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:48.670194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:48.670194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:13:48.670194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:48.670194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:13:48.670194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:48.670194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:48.670194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:48.670194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:48.670194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:48.670194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:48.670194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:48.670194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:48.670194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:48.670194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:48.670194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:48.670194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:48.670194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:48.671314 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:48.671314 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:48.671314 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:13:48.671314 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:13:48.671314 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:13:48.671314 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:13:48.671314 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:13:48.671314 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:13:48.671314 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 16:13:48.671314 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:48.673643 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 314/623] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:13:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:48.675184 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effea631-9109-4c09-875f-1f3498b79495 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:48.675655 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effea631-9109-4c09-875f-1f3498b79495 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:48.676059 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effea631-9109-4c09-875f-1f3498b79495 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:48.677691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 315/624] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:48.680347 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effea631-9109-4c09-875f-1f3498b79495 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:48.689991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8956f-994b-4d0f-9333-2f920528bb2f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:48.725673 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b27ef-12ab-4618-beda-1886b3876bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:48.726233 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b27ef-12ab-4618-beda-1886b3876bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:48.726589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b27ef-12ab-4618-beda-1886b3876bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:48.726945 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b27ef-12ab-4618-beda-1886b3876bb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:48.737618 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b27ef-12ab-4618-beda-1886b3876bb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:48.739313 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b27ef-12ab-4618-beda-1886b3876bb9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:48.763025 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-877b27ef-12ab-4618-beda-1886b3876bb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:48.763025 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:48.763025 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:48.763025 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:48.763025 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:48.763025 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:48.763025 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:48.763025 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:48.763025 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:48.763025 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:48.763025 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:48.763025 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:48.763025 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:48.763025 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:48.763025 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:48.764063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:48.764063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:48.764063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:48.764063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:48.764063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:48.764063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:48.764063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:48.764063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:48.764063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:48.764063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:48.764063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:48.764063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:48.764063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:48.764063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:48.764063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:48.764063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:48.764063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:48.764934 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:48.764934 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:48.764934 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:48.764934 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:48.769367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 310/625] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:48.770667 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599290f-30a2-45e4-a099-250aeea9f8ad None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:48.770997 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599290f-30a2-45e4-a099-250aeea9f8ad None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:48.771278 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599290f-30a2-45e4-a099-250aeea9f8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:48.777947 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:48.792385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0599290f-30a2-45e4-a099-250aeea9f8ad None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:48.796633 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0599290f-30a2-45e4-a099-250aeea9f8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:48.800117 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effea631-9109-4c09-875f-1f3498b79495 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:48.820699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:48.841223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fdd75-a0b9-4237-b458-7b99a393efb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:48.842438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fdd75-a0b9-4237-b458-7b99a393efb2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:48.842924 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fdd75-a0b9-4237-b458-7b99a393efb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:48.843415 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425fdd75-a0b9-4237-b458-7b99a393efb2 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:48.844527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425fdd75-a0b9-4237-b458-7b99a393efb2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:48.845172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425fdd75-a0b9-4237-b458-7b99a393efb2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:48.846856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425fdd75-a0b9-4237-b458-7b99a393efb2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:48.869100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 316/626] 10.222.0.67 () {56 vars in 1135 bytes} [Thu Jun 30 16:13:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:48.870305 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fdd75-a0b9-4237-b458-7b99a393efb2 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:48.887615 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 311/627] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:48.889154 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599290f-30a2-45e4-a099-250aeea9f8ad None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:49.012524 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599290f-30a2-45e4-a099-250aeea9f8ad None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:49.014089 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425fdd75-a0b9-4237-b458-7b99a393efb2 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:49.049371 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:49.057137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31833bb5-e2fc-40a8-8d90-b2fc31d3b96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:49.058268 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31833bb5-e2fc-40a8-8d90-b2fc31d3b96e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:49.063581 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31833bb5-e2fc-40a8-8d90-b2fc31d3b96e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:49.063581 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31833bb5-e2fc-40a8-8d90-b2fc31d3b96e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:49.063818 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f9a5d-4823-42d8-a956-0aa5decb33d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:49.063818 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f9a5d-4823-42d8-a956-0aa5decb33d1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:49.063818 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f9a5d-4823-42d8-a956-0aa5decb33d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:49.063818 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f9a5d-4823-42d8-a956-0aa5decb33d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:49.064094 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f9a5d-4823-42d8-a956-0aa5decb33d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:49.064094 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f9a5d-4823-42d8-a956-0aa5decb33d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:49.064354 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f9a5d-4823-42d8-a956-0aa5decb33d1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:49.072681 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31833bb5-e2fc-40a8-8d90-b2fc31d3b96e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:49.074864 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31833bb5-e2fc-40a8-8d90-b2fc31d3b96e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:49.083623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 317/628] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:49.085430 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f9a5d-4823-42d8-a956-0aa5decb33d1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:49.101538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-31833bb5-e2fc-40a8-8d90-b2fc31d3b96e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:49.101538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:49.101538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:49.101538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:49.101538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:49.101538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:49.101538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:49.101538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:49.101538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:49.101538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:49.101538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:49.101538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:49.101538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:49.101538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:49.101538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:49.102477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:49.102477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:49.102477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:49.102477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:49.102477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:49.102477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:49.102477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:49.102477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:49.102477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:49.102477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:49.102477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:49.102477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:49.102477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:49.102477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:49.102477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:49.102477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:49.102477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:49.103364 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:49.103364 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:49.103364 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:49.103364 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:49.105736 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 312/629] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:49.106775 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31833bb5-e2fc-40a8-8d90-b2fc31d3b96e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:49.140254 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:49.219332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f9a5d-4823-42d8-a956-0aa5decb33d1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:49.232981 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31833bb5-e2fc-40a8-8d90-b2fc31d3b96e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:49.259704 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da5b6f-7887-4f80-a8c8-c81d854baa9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:49.260894 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da5b6f-7887-4f80-a8c8-c81d854baa9b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:49.261388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da5b6f-7887-4f80-a8c8-c81d854baa9b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:49.261867 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da5b6f-7887-4f80-a8c8-c81d854baa9b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:49.274553 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da5b6f-7887-4f80-a8c8-c81d854baa9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:49.276119 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bc753-f56a-486e-8551-214d4c08e750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:49.276910 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bc753-f56a-486e-8551-214d4c08e750 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:49.277193 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bc753-f56a-486e-8551-214d4c08e750 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:49.277466 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4bc753-f56a-486e-8551-214d4c08e750 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:49.283836 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-42da5b6f-7887-4f80-a8c8-c81d854baa9b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:49.283836 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:49.283836 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:49.283836 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:49.283836 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:49.283836 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:49.283836 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:49.283836 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:49.283836 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:49.283836 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:49.283836 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:49.283836 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:49.283836 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:49.283836 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:49.283836 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:49.283836 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:49.284820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:49.284820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:49.284820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:49.284820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:49.284820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:49.284820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:49.284820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:49.284820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:49.284820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:49.284820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:49.284820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:49.284820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:49.284820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:49.284820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:49.284820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:49.284820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:49.285655 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:49.285655 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:49.285655 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:49.285655 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:49.285655 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:49.285655 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:49.285655 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:49.285655 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:49.285655 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:49.285655 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:49.285655 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:49.285655 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:49.286298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4bc753-f56a-486e-8551-214d4c08e750 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:49.286298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4bc753-f56a-486e-8551-214d4c08e750 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:49.293867 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 318/630] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 30 16:13:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:49.294993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da5b6f-7887-4f80-a8c8-c81d854baa9b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:49.301428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-3d4bc753-f56a-486e-8551-214d4c08e750 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:49.301428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:49.301428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:49.301428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:49.301428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:49.301428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:49.301428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:49.301428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:49.301428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:49.301428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:49.301428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:49.301428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:49.301428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:13:49.301428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:13:49.301428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:13:49.302338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:13:49.302338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:49.302338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:49.302338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:49.302338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:49.302338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:49.302338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:49.302338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:49.302338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:49.302338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:49.302338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:49.302338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:49.302338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:49.302338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:49.302338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:49.302338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:49.302338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:49.303562 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:13:49.303562 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:13:49.303562 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:13:49.303562 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:13:49.303562 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:13:49.303562 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:13:49.303562 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:49.303562 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:49.310090 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 313/631] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:13:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:49.310090 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bc753-f56a-486e-8551-214d4c08e750 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:49.358084 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:49.402972 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:49.409633 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da5b6f-7887-4f80-a8c8-c81d854baa9b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:49.419017 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4bc753-f56a-486e-8551-214d4c08e750 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:49.444945 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb1da1-dfaf-495c-a5ac-a19db5810d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:49.445581 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb1da1-dfaf-495c-a5ac-a19db5810d8b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:49.446111 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb1da1-dfaf-495c-a5ac-a19db5810d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:49.452331 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb1da1-dfaf-495c-a5ac-a19db5810d8b None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:49.452331 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb1da1-dfaf-495c-a5ac-a19db5810d8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:49.452331 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb1da1-dfaf-495c-a5ac-a19db5810d8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:49.452331 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effb1da1-dfaf-495c-a5ac-a19db5810d8b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:49.455557 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0850e-de2a-41b6-bb81-07a392ddc792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:49.456401 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0850e-de2a-41b6-bb81-07a392ddc792 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:49.456687 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0850e-de2a-41b6-bb81-07a392ddc792 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:49.456961 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0850e-de2a-41b6-bb81-07a392ddc792 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:49.457712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e0850e-de2a-41b6-bb81-07a392ddc792 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:49.458021 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e0850e-de2a-41b6-bb81-07a392ddc792 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:49.460050 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e0850e-de2a-41b6-bb81-07a392ddc792 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:49.462138 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 319/632] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:49.463427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8da18-8806-4ca6-a318-763a0c72d632 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:49.463745 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8da18-8806-4ca6-a318-763a0c72d632 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:49.464021 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8da18-8806-4ca6-a318-763a0c72d632 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:49.467611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 320/633] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:49.467611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8da18-8806-4ca6-a318-763a0c72d632 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:49.476670 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 314/634] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 30 16:13:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:49.477727 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0850e-de2a-41b6-bb81-07a392ddc792 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:49.578447 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0850e-de2a-41b6-bb81-07a392ddc792 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:49.579670 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8da18-8806-4ca6-a318-763a0c72d632 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:49.586421 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:49.614133 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fdc6d-f7da-4938-ac8f-c00dccfa9d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:49.614936 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fdc6d-f7da-4938-ac8f-c00dccfa9d6b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:49.615052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79641d-fa37-4695-900f-e84431dc163e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:49.615211 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fdc6d-f7da-4938-ac8f-c00dccfa9d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:49.615533 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5fdc6d-f7da-4938-ac8f-c00dccfa9d6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:49.616139 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79641d-fa37-4695-900f-e84431dc163e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:49.616427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79641d-fa37-4695-900f-e84431dc163e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:49.616702 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79641d-fa37-4695-900f-e84431dc163e None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:49.617483 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b79641d-fa37-4695-900f-e84431dc163e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:49.617788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b79641d-fa37-4695-900f-e84431dc163e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:49.619226 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b79641d-fa37-4695-900f-e84431dc163e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:49.622551 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5fdc6d-f7da-4938-ac8f-c00dccfa9d6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:49.624376 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5fdc6d-f7da-4938-ac8f-c00dccfa9d6b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:49.630648 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 321/635] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:13:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:49.633114 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba3b6b-18ea-4a32-b657-187acd725457 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:49.633405 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba3b6b-18ea-4a32-b657-187acd725457 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:49.633681 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdba3b6b-18ea-4a32-b657-187acd725457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:49.640019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-2b5fdc6d-f7da-4938-ac8f-c00dccfa9d6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:49.640019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:49.640019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:49.640019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:49.640019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:49.640019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:49.640019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:49.640019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:49.640019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:49.640019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:49.640019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:49.640019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:49.640019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:13:49.640019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:13:49.640019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:13:49.641082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:13:49.641082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:49.641082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:49.641082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:49.641082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:49.641082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:49.641082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:49.641082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:49.641082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:49.641082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:49.641082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:49.641082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:49.641082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:49.641082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:49.641082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:49.641082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:49.641082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:49.642065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:13:49.642065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:13:49.642065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:13:49.642065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:13:49.642065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:13:49.642065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:13:49.642065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:49.642065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:49.642065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 315/636] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:13:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:49.642731 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fdc6d-f7da-4938-ac8f-c00dccfa9d6b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:49.651814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-bdba3b6b-18ea-4a32-b657-187acd725457 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:49.653129 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bdba3b6b-18ea-4a32-b657-187acd725457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:49.663869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:49.720226 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 322/637] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:49.739699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2f126-3e76-4283-a035-94c97486007b None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:49.739699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2f126-3e76-4283-a035-94c97486007b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:49.739699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2f126-3e76-4283-a035-94c97486007b None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:49.739699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 323/638] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:49.739699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2f126-3e76-4283-a035-94c97486007b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:49.751283 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5fdc6d-f7da-4938-ac8f-c00dccfa9d6b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:49.787793 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a408d16-0995-4e48-81e8-276cd85720e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:49.788601 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a408d16-0995-4e48-81e8-276cd85720e4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:49.788881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a408d16-0995-4e48-81e8-276cd85720e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:49.789148 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a408d16-0995-4e48-81e8-276cd85720e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:49.799313 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a408d16-0995-4e48-81e8-276cd85720e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:49.800864 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a408d16-0995-4e48-81e8-276cd85720e4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:49.824851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-5a408d16-0995-4e48-81e8-276cd85720e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:49.824851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:49.824851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:49.824851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:49.824851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:49.824851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:49.824851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:49.824851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:49.824851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:49.824851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:49.824851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:49.824851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:49.824851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:49.824851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:49.824851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:49.825980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:49.825980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:49.825980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:49.825980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:49.825980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:49.825980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:49.825980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:49.825980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:49.825980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:49.825980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:49.825980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:49.825980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:49.825980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:49.825980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:49.825980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:49.825980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:49.825980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:49.826906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:49.826906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:49.826906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:49.826906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:49.826906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 316/639] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:49.826906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a408d16-0995-4e48-81e8-276cd85720e4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:49.829329 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2f126-3e76-4283-a035-94c97486007b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:49.847904 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:49.865245 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e521005-898c-4ec1-b94f-572cf1e3a451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:49.865893 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e521005-898c-4ec1-b94f-572cf1e3a451 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:49.866180 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e521005-898c-4ec1-b94f-572cf1e3a451 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:49.866460 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e521005-898c-4ec1-b94f-572cf1e3a451 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:49.867321 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e521005-898c-4ec1-b94f-572cf1e3a451 None admin] RBAC: Adding query filter params (role.id=57bc5df632b3407d97116ae34d92a5cb, scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d, user.id=71cca3708386459197abf6c67505bc5f) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:49.867673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e521005-898c-4ec1-b94f-572cf1e3a451 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:49.869178 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e521005-898c-4ec1-b94f-572cf1e3a451 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:49.875137 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 324/640] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:13:49 2022] GET /identity/v3/role_assignments?user.id=71cca3708386459197abf6c67505bc5f&scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d&role.id=57bc5df632b3407d97116ae34d92a5cb => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:49.876406 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af9d96-bae3-4983-9eb6-e8e0d79928a6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:49.876673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af9d96-bae3-4983-9eb6-e8e0d79928a6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:49.876957 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af9d96-bae3-4983-9eb6-e8e0d79928a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:49.894727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b5af9d96-bae3-4983-9eb6-e8e0d79928a6 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:49.896003 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b5af9d96-bae3-4983-9eb6-e8e0d79928a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:49.900509 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:49.942209 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a408d16-0995-4e48-81e8-276cd85720e4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:49.984652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da15ab-ba42-47a2-b1a9-14ce90dcfa68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:49.985406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da15ab-ba42-47a2-b1a9-14ce90dcfa68 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:49.985561 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 325/641] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:49.985677 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da15ab-ba42-47a2-b1a9-14ce90dcfa68 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:49.985877 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da15ab-ba42-47a2-b1a9-14ce90dcfa68 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:49.986837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da15ab-ba42-47a2-b1a9-14ce90dcfa68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:49.986951 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af9d96-bae3-4983-9eb6-e8e0d79928a6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:49.987220 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da15ab-ba42-47a2-b1a9-14ce90dcfa68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:49.988787 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da15ab-ba42-47a2-b1a9-14ce90dcfa68 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:50.015722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 317/642] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:13:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:50.015722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da15ab-ba42-47a2-b1a9-14ce90dcfa68 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:50.117292 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:50.123842 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af9d96-bae3-4983-9eb6-e8e0d79928a6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:50.143016 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da15ab-ba42-47a2-b1a9-14ce90dcfa68 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:50.166064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0542b-4246-4262-99c9-cf4866ed716c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:50.166858 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0542b-4246-4262-99c9-cf4866ed716c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:50.167724 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0542b-4246-4262-99c9-cf4866ed716c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:50.168013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0542b-4246-4262-99c9-cf4866ed716c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:50.180188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:50.180918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f0542b-4246-4262-99c9-cf4866ed716c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:50.186233 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e2b78-afb1-4e32-856c-644db5676ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:50.186903 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e2b78-afb1-4e32-856c-644db5676ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:50.187185 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e2b78-afb1-4e32-856c-644db5676ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:50.187507 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e2b78-afb1-4e32-856c-644db5676ba3 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:50.188285 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e2b78-afb1-4e32-856c-644db5676ba3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:50.188590 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e2b78-afb1-4e32-856c-644db5676ba3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:50.188824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-37f0542b-4246-4262-99c9-cf4866ed716c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:50.188824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:50.188824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:50.188824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:50.188824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:50.188824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:50.188824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:50.188824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:50.188824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:50.188824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:50.188824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:50.188824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:50.188824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:50.188824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:50.188824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:50.188824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:50.189755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:50.189755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:50.189755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:50.189755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:50.189755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:50.189755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:50.189755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:50.189755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:50.189755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:50.189755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:50.189755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:50.189755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:50.189755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:50.189755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:50.189755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:50.189755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:50.190577 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:50.190577 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:50.190577 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:50.190577 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:50.190577 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:50.190577 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:50.190577 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:50.190577 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:50.190577 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:50.190577 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:50.190577 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:50.190577 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:50.191195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e2b78-afb1-4e32-856c-644db5676ba3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:50.191273 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 326/643] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 30 16:13:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:50.191273 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0542b-4246-4262-99c9-cf4866ed716c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:50.213511 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 318/644] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:50.213511 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e2b78-afb1-4e32-856c-644db5676ba3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:50.307620 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0542b-4246-4262-99c9-cf4866ed716c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:50.321802 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e2b78-afb1-4e32-856c-644db5676ba3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:50.345183 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e770b4-d88a-4926-9e1e-8bc7ee2da8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:50.346338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e770b4-d88a-4926-9e1e-8bc7ee2da8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:50.346771 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e770b4-d88a-4926-9e1e-8bc7ee2da8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:50.347220 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e770b4-d88a-4926-9e1e-8bc7ee2da8f8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:50.358606 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e770b4-d88a-4926-9e1e-8bc7ee2da8f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:50.358910 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ba802-4c03-44db-aadf-ab4a61d05ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:50.359854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ba802-4c03-44db-aadf-ab4a61d05ef2 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:50.360139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ba802-4c03-44db-aadf-ab4a61d05ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:50.360415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ba802-4c03-44db-aadf-ab4a61d05ef2 None admin] PATH_INFO: `/v3/projects/19ef92ebcce94b90bf39ad3a58772a1d/users/988b917eea4b48ffae32e23b8555a5c6/roles/57bc5df632b3407d97116ae34d92a5cb` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:50.368560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-34e770b4-d88a-4926-9e1e-8bc7ee2da8f8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:50.368560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:50.368560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:50.368560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:50.368560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:50.368560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:50.368560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:50.368560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:50.368560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:50.368560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:50.368560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:50.368560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:50.368560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:50.368560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:50.368560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:50.368560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:50.369666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:50.369666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:50.369666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:50.369666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:50.369666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:50.369666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:50.369666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:50.369666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:50.369666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:50.369666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:50.369666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:50.369666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:50.369666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:50.369666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:50.369666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:50.369666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:50.370829 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:50.370829 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:50.370829 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:50.370829 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:50.370829 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:50.370829 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:50.370829 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:50.370829 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:50.370829 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:50.370829 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:50.370829 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:50.370829 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:50.371642 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:50.372837 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 327/645] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 30 16:13:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:50.373650 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e770b4-d88a-4926-9e1e-8bc7ee2da8f8 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:50.381635 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ba802-4c03-44db-aadf-ab4a61d05ef2 None admin] RBAC: Authorizing `identity:create_grant(project_id=19ef92ebcce94b90bf39ad3a58772a1d, user_id=988b917eea4b48ffae32e23b8555a5c6, role_id=57bc5df632b3407d97116ae34d92a5cb)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:50.383772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ba802-4c03-44db-aadf-ab4a61d05ef2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:50.438078 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:50.452555 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 319/646] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 30 16:13:50 2022] PUT /identity/v3/projects/19ef92ebcce94b90bf39ad3a58772a1d/users/988b917eea4b48ffae32e23b8555a5c6/roles/57bc5df632b3407d97116ae34d92a5cb => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:13:50.464830 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ba802-4c03-44db-aadf-ab4a61d05ef2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:50.492588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e770b4-d88a-4926-9e1e-8bc7ee2da8f8 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:50.549606 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474006d-330a-44e6-b000-54179c2020a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:50.550594 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474006d-330a-44e6-b000-54179c2020a1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:50.550879 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474006d-330a-44e6-b000-54179c2020a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:50.551154 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474006d-330a-44e6-b000-54179c2020a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:50.552437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474006d-330a-44e6-b000-54179c2020a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:50.554377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474006d-330a-44e6-b000-54179c2020a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:50.555853 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474006d-330a-44e6-b000-54179c2020a1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:50.575646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 328/647] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 30 16:13:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:50.576857 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474006d-330a-44e6-b000-54179c2020a1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:50.604294 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ba802-4c03-44db-aadf-ab4a61d05ef2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:50.649136 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8af57b-ea24-4ca8-8c36-c96d8a9b81ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:50.649858 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8af57b-ea24-4ca8-8c36-c96d8a9b81ba None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:50.650139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8af57b-ea24-4ca8-8c36-c96d8a9b81ba None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:50.650416 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8af57b-ea24-4ca8-8c36-c96d8a9b81ba None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:50.659331 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8af57b-ea24-4ca8-8c36-c96d8a9b81ba None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:50.660813 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8af57b-ea24-4ca8-8c36-c96d8a9b81ba None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:50.679206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-4f8af57b-ea24-4ca8-8c36-c96d8a9b81ba None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 16:13:50.679206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:50.679206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:50.679206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:50.679206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:50.679206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:50.679206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:50.679206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:50.679206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:50.679206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:50.679206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:50.679206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:50.679206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:13:50.679206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:13:50.679206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:13:50.679206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:13:50.680426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:50.680426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:50.680426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:13:50.680426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:50.680426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:13:50.680426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:50.680426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:50.680426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:50.680426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:50.680426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:50.680426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:50.680426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:50.680426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:50.680426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:50.680426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:50.680426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:50.680426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:50.680426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:50.681530 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:50.681530 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:50.681530 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:13:50.681530 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:13:50.681530 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:13:50.681530 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:13:50.681530 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:13:50.681530 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:13:50.681530 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 16:13:50.681530 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:50.681530 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 320/648] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:50.682203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8af57b-ea24-4ca8-8c36-c96d8a9b81ba None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:50.712168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474006d-330a-44e6-b000-54179c2020a1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:50.737366 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:50.744893 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:50.759731 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08903574-fc39-474f-8a69-4bded2bfdbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:50.761536 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08903574-fc39-474f-8a69-4bded2bfdbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:50.761977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08903574-fc39-474f-8a69-4bded2bfdbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:50.762418 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08903574-fc39-474f-8a69-4bded2bfdbe1 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:50.763317 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08903574-fc39-474f-8a69-4bded2bfdbe1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:50.763792 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08903574-fc39-474f-8a69-4bded2bfdbe1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:50.765341 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08903574-fc39-474f-8a69-4bded2bfdbe1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:50.779173 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 329/649] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 30 16:13:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:50.780240 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08903574-fc39-474f-8a69-4bded2bfdbe1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:50.800541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8af57b-ea24-4ca8-8c36-c96d8a9b81ba None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:50.839143 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb2bb2-95dc-4ef3-9b68-26062ceaf781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:50.839941 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb2bb2-95dc-4ef3-9b68-26062ceaf781 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:50.840294 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb2bb2-95dc-4ef3-9b68-26062ceaf781 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:50.840643 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb2bb2-95dc-4ef3-9b68-26062ceaf781 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:50.855421 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb2bb2-95dc-4ef3-9b68-26062ceaf781 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:50.857445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb2bb2-95dc-4ef3-9b68-26062ceaf781 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:50.878125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-0fbb2bb2-95dc-4ef3-9b68-26062ceaf781 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:50.878125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:50.878125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:50.878125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:50.878125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:50.878125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:50.878125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:50.878125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:50.878125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:50.878125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:50.878125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:50.878125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:50.878125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:50.878125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:50.878125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:50.879297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:50.879297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:50.879297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:50.879297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:50.879297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:50.879297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:50.879297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:50.879297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:50.879297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:50.879297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:50.879297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:50.879297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:50.879297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:50.879297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:50.879297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:50.879297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:50.879297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:50.880433 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:50.880433 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:50.880433 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:50.880433 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:50.885484 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 321/650] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:50.886428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb2bb2-95dc-4ef3-9b68-26062ceaf781 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:50.910852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08903574-fc39-474f-8a69-4bded2bfdbe1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:50.953672 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d7796-2080-43fc-b162-56f4d1d8b5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:50.954856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d7796-2080-43fc-b162-56f4d1d8b5bc None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:50.955288 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d7796-2080-43fc-b162-56f4d1d8b5bc None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:50.955866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d7796-2080-43fc-b162-56f4d1d8b5bc None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:50.956754 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d7796-2080-43fc-b162-56f4d1d8b5bc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:50.957075 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d7796-2080-43fc-b162-56f4d1d8b5bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:50.960073 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153d7796-2080-43fc-b162-56f4d1d8b5bc None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:50.985899 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 330/651] 10.222.0.67 () {56 vars in 1137 bytes} [Thu Jun 30 16:13:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:50.987280 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d7796-2080-43fc-b162-56f4d1d8b5bc None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:51.007784 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:51.011757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb2bb2-95dc-4ef3-9b68-26062ceaf781 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:51.043871 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:51.051532 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8bed4-f3c3-4fb7-935d-ba9dcf51101b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:51.052625 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8bed4-f3c3-4fb7-935d-ba9dcf51101b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:51.053596 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8bed4-f3c3-4fb7-935d-ba9dcf51101b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:51.054158 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8bed4-f3c3-4fb7-935d-ba9dcf51101b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:51.066832 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8bed4-f3c3-4fb7-935d-ba9dcf51101b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:51.083670 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f8bed4-f3c3-4fb7-935d-ba9dcf51101b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:51.091721 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-39f8bed4-f3c3-4fb7-935d-ba9dcf51101b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:51.091721 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:51.091721 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:51.091721 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:51.091721 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:51.091721 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:51.091721 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:51.091721 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:51.091721 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:51.091721 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:51.091721 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:51.091721 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:51.091721 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:51.091721 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:51.091721 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:51.092631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:51.092631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:51.092631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:51.092631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:51.092631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:51.092631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:51.092631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:51.092631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:51.092631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:51.092631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:51.092631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:51.092631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:51.092631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:51.092631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:51.092631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:51.092631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:51.092631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:51.093561 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:51.093561 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:51.093561 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:51.093561 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:51.100910 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 322/652] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:51.102302 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8bed4-f3c3-4fb7-935d-ba9dcf51101b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:51.109579 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d7796-2080-43fc-b162-56f4d1d8b5bc None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:51.152784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e33ba-2a11-4487-845d-f9d993afcc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:51.153631 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e33ba-2a11-4487-845d-f9d993afcc35 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:51.153850 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e33ba-2a11-4487-845d-f9d993afcc35 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:51.154087 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e33ba-2a11-4487-845d-f9d993afcc35 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:51.154817 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e33ba-2a11-4487-845d-f9d993afcc35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:51.155119 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e33ba-2a11-4487-845d-f9d993afcc35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:51.156725 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e33ba-2a11-4487-845d-f9d993afcc35 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:51.176546 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 331/653] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:51.177809 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e33ba-2a11-4487-845d-f9d993afcc35 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:51.223631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8bed4-f3c3-4fb7-935d-ba9dcf51101b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:51.268519 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2971bd2-6b1a-45eb-adbb-069febd875d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:51.269344 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2971bd2-6b1a-45eb-adbb-069febd875d4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:51.269622 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2971bd2-6b1a-45eb-adbb-069febd875d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:51.269893 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2971bd2-6b1a-45eb-adbb-069febd875d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:51.291603 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e33ba-2a11-4487-845d-f9d993afcc35 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:51.293664 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:51.298916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:51.336010 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f50a8b-778e-45e6-9894-0cc749372f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:51.336945 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f50a8b-778e-45e6-9894-0cc749372f04 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:51.337232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f50a8b-778e-45e6-9894-0cc749372f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:51.337514 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f50a8b-778e-45e6-9894-0cc749372f04 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:51.338231 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f50a8b-778e-45e6-9894-0cc749372f04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:51.338538 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f50a8b-778e-45e6-9894-0cc749372f04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:51.340565 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f50a8b-778e-45e6-9894-0cc749372f04 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:51.353154 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2971bd2-6b1a-45eb-adbb-069febd875d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:51.359779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 332/654] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:51.359779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f50a8b-778e-45e6-9894-0cc749372f04 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:51.360895 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2971bd2-6b1a-45eb-adbb-069febd875d4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:51.464369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f50a8b-778e-45e6-9894-0cc749372f04 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:51.470479 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 323/655] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:13:51.471708 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2971bd2-6b1a-45eb-adbb-069febd875d4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:51.501581 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9087a0-9a2c-4772-a7cd-d99dc95f028d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:51.502379 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9087a0-9a2c-4772-a7cd-d99dc95f028d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:51.502660 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9087a0-9a2c-4772-a7cd-d99dc95f028d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:51.502933 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9087a0-9a2c-4772-a7cd-d99dc95f028d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:51.511244 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9087a0-9a2c-4772-a7cd-d99dc95f028d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:51.512994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9087a0-9a2c-4772-a7cd-d99dc95f028d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:51.528552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-7b9087a0-9a2c-4772-a7cd-d99dc95f028d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 16:13:51.528552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:51.528552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:51.528552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:51.528552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:51.528552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:51.528552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:51.528552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:51.528552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:51.528552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:51.528552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:51.528552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:51.528552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:13:51.528552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:13:51.528552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:13:51.528552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:13:51.529549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:51.529549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:51.529549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:13:51.529549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:51.529549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:13:51.529549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:51.529549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:51.529549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:51.529549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:51.529549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:51.529549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:51.529549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:51.529549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:51.529549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:51.529549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:51.529549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:51.529549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:51.529549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:51.530552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:51.530552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:51.530552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:13:51.530552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:13:51.530552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:13:51.530552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:13:51.530552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:13:51.530552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:13:51.530552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 16:13:51.530552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:51.534324 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 333/656] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:51.535230 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9087a0-9a2c-4772-a7cd-d99dc95f028d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:51.536513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:51.561315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:51.590273 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2971bd2-6b1a-45eb-adbb-069febd875d4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:51.628146 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a395ff-81fe-47aa-9584-84aa76285a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:51.629406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a395ff-81fe-47aa-9584-84aa76285a1c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:51.630039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a395ff-81fe-47aa-9584-84aa76285a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:51.630521 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a395ff-81fe-47aa-9584-84aa76285a1c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:51.639823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a395ff-81fe-47aa-9584-84aa76285a1c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:51.641636 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a395ff-81fe-47aa-9584-84aa76285a1c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:51.642098 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9087a0-9a2c-4772-a7cd-d99dc95f028d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:51.659912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-80a395ff-81fe-47aa-9584-84aa76285a1c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 16:13:51.659912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:51.659912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:51.659912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:51.659912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:51.659912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:51.659912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:51.659912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:51.659912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:51.659912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:51.659912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:51.659912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:51.659912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:13:51.659912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:13:51.659912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:13:51.659912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:13:51.660949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:51.660949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:51.660949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:13:51.660949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:51.660949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:13:51.660949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:51.660949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:51.660949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:51.660949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:51.660949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:51.660949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:51.660949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:51.660949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:51.660949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:51.660949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:51.660949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:51.660949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:51.660949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:51.661951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:51.661951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:51.661951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:13:51.661951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:13:51.661951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:13:51.661951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:13:51.661951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:13:51.661951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:13:51.661951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 16:13:51.661951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:51.665193 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 324/657] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 30 16:13:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:51.666547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a395ff-81fe-47aa-9584-84aa76285a1c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:51.681261 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266d542-53e7-4b12-a1fa-28e4fd5e8f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:51.682023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266d542-53e7-4b12-a1fa-28e4fd5e8f07 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:51.682283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266d542-53e7-4b12-a1fa-28e4fd5e8f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:51.682554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5266d542-53e7-4b12-a1fa-28e4fd5e8f07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:51.693227 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266d542-53e7-4b12-a1fa-28e4fd5e8f07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:51.694722 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5266d542-53e7-4b12-a1fa-28e4fd5e8f07 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:51.713222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-5266d542-53e7-4b12-a1fa-28e4fd5e8f07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:51.713222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:51.713222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:51.713222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:51.713222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:51.713222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:51.713222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:51.713222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:51.713222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:51.713222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:51.713222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:51.713222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:51.713222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:51.713222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:51.713222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:51.714201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:51.714201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:51.714201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:51.714201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:51.714201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:51.714201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:51.714201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:51.714201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:51.714201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:51.714201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:51.714201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:51.714201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:51.714201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:51.714201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:51.714201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:51.714201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:51.714201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:51.715160 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:51.715160 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:51.715160 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:51.715160 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:51.715160 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 334/658] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:51.715926 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266d542-53e7-4b12-a1fa-28e4fd5e8f07 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:51.786626 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a395ff-81fe-47aa-9584-84aa76285a1c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:51.792866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:51.802524 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:51.819737 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5266d542-53e7-4b12-a1fa-28e4fd5e8f07 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:51.826900 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee874699-5021-4ac4-a179-3bbe334952e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:51.827895 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee874699-5021-4ac4-a179-3bbe334952e2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:51.828432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee874699-5021-4ac4-a179-3bbe334952e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:51.828928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee874699-5021-4ac4-a179-3bbe334952e2 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:51.829994 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee874699-5021-4ac4-a179-3bbe334952e2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:51.830510 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee874699-5021-4ac4-a179-3bbe334952e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:51.832223 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee874699-5021-4ac4-a179-3bbe334952e2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:51.853873 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 325/659] 10.222.0.67 () {56 vars in 1137 bytes} [Thu Jun 30 16:13:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:51.855244 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee874699-5021-4ac4-a179-3bbe334952e2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:51.859718 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe395b5-c5f8-43b9-b60b-fa0f0c72c05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:51.859886 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe395b5-c5f8-43b9-b60b-fa0f0c72c05c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:51.860173 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe395b5-c5f8-43b9-b60b-fa0f0c72c05c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:51.860438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe395b5-c5f8-43b9-b60b-fa0f0c72c05c None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:51.861280 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe395b5-c5f8-43b9-b60b-fa0f0c72c05c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:51.861590 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe395b5-c5f8-43b9-b60b-fa0f0c72c05c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:51.863118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe395b5-c5f8-43b9-b60b-fa0f0c72c05c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:51.882733 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 335/660] 10.222.0.67 () {56 vars in 1139 bytes} [Thu Jun 30 16:13:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:51.883299 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe395b5-c5f8-43b9-b60b-fa0f0c72c05c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:51.973419 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee874699-5021-4ac4-a179-3bbe334952e2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:51.985713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe395b5-c5f8-43b9-b60b-fa0f0c72c05c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:52.017043 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:52.019318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eaefc3-57ab-4fb7-9abb-d0879b8ecc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:52.020674 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eaefc3-57ab-4fb7-9abb-d0879b8ecc96 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:52.021164 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eaefc3-57ab-4fb7-9abb-d0879b8ecc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:52.021746 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eaefc3-57ab-4fb7-9abb-d0879b8ecc96 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:52.022129 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52a0dd-46c6-4a47-bd71-7c2ee1f9be85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:52.022866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52a0dd-46c6-4a47-bd71-7c2ee1f9be85 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:52.023138 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eaefc3-57ab-4fb7-9abb-d0879b8ecc96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:52.023253 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52a0dd-46c6-4a47-bd71-7c2ee1f9be85 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:52.023414 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52a0dd-46c6-4a47-bd71-7c2ee1f9be85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:52.024206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eaefc3-57ab-4fb7-9abb-d0879b8ecc96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:52.025913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eaefc3-57ab-4fb7-9abb-d0879b8ecc96 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:52.039793 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 326/661] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:52.040942 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eaefc3-57ab-4fb7-9abb-d0879b8ecc96 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:52.074901 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:52.098034 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c52a0dd-46c6-4a47-bd71-7c2ee1f9be85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:52.100204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c52a0dd-46c6-4a47-bd71-7c2ee1f9be85 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:52.154918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eaefc3-57ab-4fb7-9abb-d0879b8ecc96 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:52.193942 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf887926-2dcd-452d-a3e3-155b8b3105e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:52.194582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf887926-2dcd-452d-a3e3-155b8b3105e1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:52.195128 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf887926-2dcd-452d-a3e3-155b8b3105e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:52.195128 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf887926-2dcd-452d-a3e3-155b8b3105e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:52.208994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 336/662] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:13:52.210463 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52a0dd-46c6-4a47-bd71-7c2ee1f9be85 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:52.245873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:52.276109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf887926-2dcd-452d-a3e3-155b8b3105e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:52.277671 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf887926-2dcd-452d-a3e3-155b8b3105e1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:52.308911 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:52.319888 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52a0dd-46c6-4a47-bd71-7c2ee1f9be85 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:52.361401 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8e48f-5b9a-433c-a400-4796f6a4bcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:52.362453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8e48f-5b9a-433c-a400-4796f6a4bcfc None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:52.362732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8e48f-5b9a-433c-a400-4796f6a4bcfc None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:52.363002 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c8e48f-5b9a-433c-a400-4796f6a4bcfc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:52.371597 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c8e48f-5b9a-433c-a400-4796f6a4bcfc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:52.373254 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c8e48f-5b9a-433c-a400-4796f6a4bcfc None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:52.388205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 327/663] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:13:52.388205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c615859-4126-4de2-8ba1-7e2b19ab21ac None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:52.388205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c615859-4126-4de2-8ba1-7e2b19ab21ac None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:52.388205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c615859-4126-4de2-8ba1-7e2b19ab21ac None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:52.388205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 328/664] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:52.388649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-28c8e48f-5b9a-433c-a400-4796f6a4bcfc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:52.388649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:52.388649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:52.388649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:52.388649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:52.388649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:52.388649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:52.388649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:52.388649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:52.388649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:52.388649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:52.388649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:52.388649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:13:52.388649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:13:52.388649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:13:52.389446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:13:52.389446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:52.389446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:52.389446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:52.389446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:52.389446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:52.389446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:52.389446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:52.389446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:52.389446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:52.389446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:52.389446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:52.389446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:52.389446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:52.389446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:52.389446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:52.389446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:52.391138 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:13:52.391138 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:13:52.391138 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:13:52.391138 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:13:52.391138 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:13:52.391138 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:13:52.391138 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:52.391138 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:52.391811 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c615859-4126-4de2-8ba1-7e2b19ab21ac None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:52.393923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 337/665] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:13:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:52.394725 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8e48f-5b9a-433c-a400-4796f6a4bcfc None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:52.505685 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:52.505925 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c615859-4126-4de2-8ba1-7e2b19ab21ac None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:52.515276 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c8e48f-5b9a-433c-a400-4796f6a4bcfc None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:52.539361 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:52.545810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77456394-795e-4d0b-a42a-867c45fb0747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:52.546601 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77456394-795e-4d0b-a42a-867c45fb0747 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:52.546880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77456394-795e-4d0b-a42a-867c45fb0747 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:52.547145 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77456394-795e-4d0b-a42a-867c45fb0747 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:52.556047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa40bba-27c4-487a-bf72-f9dd4e96bcb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:52.557441 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa40bba-27c4-487a-bf72-f9dd4e96bcb5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:52.557914 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa40bba-27c4-487a-bf72-f9dd4e96bcb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:52.558760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa40bba-27c4-487a-bf72-f9dd4e96bcb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:52.559614 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77456394-795e-4d0b-a42a-867c45fb0747 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:52.561130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77456394-795e-4d0b-a42a-867c45fb0747 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:52.570542 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa40bba-27c4-487a-bf72-f9dd4e96bcb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:52.572234 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa40bba-27c4-487a-bf72-f9dd4e96bcb5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:52.579139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-77456394-795e-4d0b-a42a-867c45fb0747 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:52.579139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:52.579139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:52.579139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:52.579139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:52.579139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:52.579139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:52.579139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:52.579139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:52.579139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:52.579139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:52.579139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:52.579139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:52.579139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:52.579139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:52.580251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:52.580251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:52.580251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:52.580251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:52.580251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:52.580251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:52.580251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:52.580251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:52.580251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:52.580251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:52.580251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:52.580251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:52.580251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:52.580251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:52.580251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:52.580251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:52.580251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:52.581334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:52.581334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:52.581334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:52.581334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:52.586521 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 329/666] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:52.587614 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77456394-795e-4d0b-a42a-867c45fb0747 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:52.595760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8aa40bba-27c4-487a-bf72-f9dd4e96bcb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:52.595760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:52.595760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:52.595760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:52.595760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:52.595760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:52.595760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:52.595760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:52.595760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:52.595760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:52.595760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:52.595760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:52.595760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:52.595760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:52.595760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:52.596646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:52.596646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:52.596646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:52.596646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:52.596646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:52.596646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:52.596646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:52.596646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:52.596646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:52.596646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:52.596646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:52.596646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:52.596646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:52.596646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:52.596646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:52.596646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:52.596646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:52.597555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:52.597555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:52.597555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:52.597555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:52.600314 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 338/667] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:52.601992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbec00f-702e-4227-a9c0-0a7a1f41b095 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:52.602529 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbec00f-702e-4227-a9c0-0a7a1f41b095 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:52.602910 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbec00f-702e-4227-a9c0-0a7a1f41b095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:52.624142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6dbec00f-702e-4227-a9c0-0a7a1f41b095 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:52.625357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6dbec00f-702e-4227-a9c0-0a7a1f41b095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:52.695544 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 339/668] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:52.696385 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbec00f-702e-4227-a9c0-0a7a1f41b095 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:52.700649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77456394-795e-4d0b-a42a-867c45fb0747 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:52.735204 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a344c5-4164-4946-be9f-bb7947c1d749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:52.735887 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a344c5-4164-4946-be9f-bb7947c1d749 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:52.736166 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a344c5-4164-4946-be9f-bb7947c1d749 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:52.736438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a344c5-4164-4946-be9f-bb7947c1d749 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:52.737203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a344c5-4164-4946-be9f-bb7947c1d749 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:52.737510 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a344c5-4164-4946-be9f-bb7947c1d749 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:52.738933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a344c5-4164-4946-be9f-bb7947c1d749 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:52.751707 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 330/669] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:13:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:52.752496 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a344c5-4164-4946-be9f-bb7947c1d749 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:52.782113 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:52.798682 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:52.806837 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbec00f-702e-4227-a9c0-0a7a1f41b095 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:52.841183 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb312fd-2a5f-4a9d-b0ee-c1af4210007c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:52.841840 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb312fd-2a5f-4a9d-b0ee-c1af4210007c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:52.842128 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb312fd-2a5f-4a9d-b0ee-c1af4210007c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:52.842405 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb312fd-2a5f-4a9d-b0ee-c1af4210007c None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:52.843123 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb312fd-2a5f-4a9d-b0ee-c1af4210007c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:52.843438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb312fd-2a5f-4a9d-b0ee-c1af4210007c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:52.844845 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb312fd-2a5f-4a9d-b0ee-c1af4210007c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:52.854889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a344c5-4164-4946-be9f-bb7947c1d749 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:52.860999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 340/670] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:52.860999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d69c7-6230-4eaf-9d5c-5444630d9617 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:52.860999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d69c7-6230-4eaf-9d5c-5444630d9617 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:52.860999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d69c7-6230-4eaf-9d5c-5444630d9617 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:52.860999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 341/671] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:52.860999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d69c7-6230-4eaf-9d5c-5444630d9617 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:52.891571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38deec4-ae98-4278-b5ac-462a7c6b3670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:52.892230 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38deec4-ae98-4278-b5ac-462a7c6b3670 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:52.892509 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38deec4-ae98-4278-b5ac-462a7c6b3670 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:52.892776 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38deec4-ae98-4278-b5ac-462a7c6b3670 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:52.893483 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38deec4-ae98-4278-b5ac-462a7c6b3670 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:52.893786 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38deec4-ae98-4278-b5ac-462a7c6b3670 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:52.895285 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38deec4-ae98-4278-b5ac-462a7c6b3670 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:52.907016 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 331/672] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:52.907906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38deec4-ae98-4278-b5ac-462a7c6b3670 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:52.967130 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d69c7-6230-4eaf-9d5c-5444630d9617 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:53.003648 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630bce0c-074d-44dc-aaed-f88e9523ddca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:53.004358 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630bce0c-074d-44dc-aaed-f88e9523ddca None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:53.004642 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630bce0c-074d-44dc-aaed-f88e9523ddca None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:53.004913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630bce0c-074d-44dc-aaed-f88e9523ddca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:53.007053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:53.012339 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38deec4-ae98-4278-b5ac-462a7c6b3670 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:53.013893 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:53.016726 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630bce0c-074d-44dc-aaed-f88e9523ddca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:53.025218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-630bce0c-074d-44dc-aaed-f88e9523ddca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:13:53.025218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:53.025218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:53.025218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:53.025218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:53.025218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:53.025218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:53.025218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:53.025218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:53.025218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:53.025218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:53.025218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:53.025218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:53.025218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:53.025218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:53.025218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:53.026168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:53.026168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:53.026168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:53.026168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:53.026168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:53.026168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:53.026168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:53.026168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:53.026168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:53.026168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:53.026168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:53.026168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:53.026168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:53.026168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:53.026168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:53.026168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:53.027100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:53.027100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:53.027100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:53.027100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:53.027100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:53.027100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:53.027100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:53.027100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:53.027100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:53.027100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:53.027100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:13:53.027100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:53.027100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 342/673] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:13:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:53.027821 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630bce0c-074d-44dc-aaed-f88e9523ddca None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:53.052323 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1868b6b5-0c54-4686-80cb-c510c09c29f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:53.053333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868b6b5-0c54-4686-80cb-c510c09c29f2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:53.053333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868b6b5-0c54-4686-80cb-c510c09c29f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:53.053500 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1868b6b5-0c54-4686-80cb-c510c09c29f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:53.054271 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868b6b5-0c54-4686-80cb-c510c09c29f2 None admin] RBAC: Adding query filter params (role.id=57bc5df632b3407d97116ae34d92a5cb, scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d, user.id=61437a2fa2a94b35b1b4a5750f6e2cad) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:53.054584 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868b6b5-0c54-4686-80cb-c510c09c29f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:53.056048 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1868b6b5-0c54-4686-80cb-c510c09c29f2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:53.062441 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 332/674] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:13:52 2022] GET /identity/v3/role_assignments?user.id=61437a2fa2a94b35b1b4a5750f6e2cad&scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d&role.id=57bc5df632b3407d97116ae34d92a5cb => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:53.063706 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0baa20-f06e-497e-99cb-9d2ef737d055 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:53.063977 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0baa20-f06e-497e-99cb-9d2ef737d055 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:53.064254 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0baa20-f06e-497e-99cb-9d2ef737d055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:53.083746 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0c0baa20-f06e-497e-99cb-9d2ef737d055 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:53.085734 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0c0baa20-f06e-497e-99cb-9d2ef737d055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:53.141355 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630bce0c-074d-44dc-aaed-f88e9523ddca None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:53.167172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 333/675] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:53.168107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0baa20-f06e-497e-99cb-9d2ef737d055 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:53.191158 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694830c-eb91-4f56-ba01-0f30286e92aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:53.192333 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694830c-eb91-4f56-ba01-0f30286e92aa None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:53.192449 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694830c-eb91-4f56-ba01-0f30286e92aa None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:53.192716 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694830c-eb91-4f56-ba01-0f30286e92aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:53.200829 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7694830c-eb91-4f56-ba01-0f30286e92aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:53.202396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7694830c-eb91-4f56-ba01-0f30286e92aa None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:53.218376 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-7694830c-eb91-4f56-ba01-0f30286e92aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:53.218376 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:53.218376 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:53.218376 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:53.218376 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:53.218376 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:53.218376 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:53.218376 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:53.218376 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:53.218376 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:53.218376 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:53.218376 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:53.218376 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:13:53.218376 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:13:53.218376 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:13:53.219769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:13:53.219769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:53.219769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:53.219769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:53.219769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:53.219769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:53.219769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:53.219769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:53.219769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:53.219769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:53.219769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:53.219769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:53.219769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:53.219769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:53.219769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:53.219769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:53.219769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:53.220654 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:13:53.220654 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:13:53.220654 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:13:53.220654 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:13:53.220654 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:13:53.220654 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:13:53.220654 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:53.220654 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:53.223694 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 343/676] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:13:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:53.224588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694830c-eb91-4f56-ba01-0f30286e92aa None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:53.291621 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:53.294635 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:53.297550 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0baa20-f06e-497e-99cb-9d2ef737d055 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:53.335824 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7dab9c-ae0a-4ce7-b37f-175ff4ecddee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:53.337162 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dab9c-ae0a-4ce7-b37f-175ff4ecddee None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:53.337586 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dab9c-ae0a-4ce7-b37f-175ff4ecddee None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:53.337868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7dab9c-ae0a-4ce7-b37f-175ff4ecddee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:53.341891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694830c-eb91-4f56-ba01-0f30286e92aa None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:53.346878 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7dab9c-ae0a-4ce7-b37f-175ff4ecddee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:53.348423 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7dab9c-ae0a-4ce7-b37f-175ff4ecddee None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:53.367319 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ae7dab9c-ae0a-4ce7-b37f-175ff4ecddee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:53.367319 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:53.367319 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:53.367319 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:53.367319 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:53.367319 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:53.367319 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:53.367319 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:53.367319 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:53.367319 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:53.367319 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:53.367319 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:53.367319 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:13:53.367319 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:13:53.367319 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:13:53.368389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:13:53.368389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:53.368389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:53.368389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:53.368389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:53.368389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:53.368389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:53.368389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:53.368389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:53.368389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:53.368389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:53.368389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:53.368389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:53.368389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:53.368389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:53.368389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:53.368389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:53.369322 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:13:53.369322 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:13:53.369322 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:13:53.369322 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:13:53.369322 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:13:53.369322 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:13:53.369322 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:53.369322 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:53.369322 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 334/677] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:13:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:53.369807 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7dab9c-ae0a-4ce7-b37f-175ff4ecddee None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:53.384362 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efa007-09a3-4801-8923-d2b288b7d55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:53.385052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efa007-09a3-4801-8923-d2b288b7d55c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:53.385340 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efa007-09a3-4801-8923-d2b288b7d55c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:53.385622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57efa007-09a3-4801-8923-d2b288b7d55c None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:53.386384 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57efa007-09a3-4801-8923-d2b288b7d55c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:53.386699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57efa007-09a3-4801-8923-d2b288b7d55c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:53.388132 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57efa007-09a3-4801-8923-d2b288b7d55c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:53.398941 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 344/678] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:53.399666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efa007-09a3-4801-8923-d2b288b7d55c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:53.483321 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7dab9c-ae0a-4ce7-b37f-175ff4ecddee None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:53.512047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57efa007-09a3-4801-8923-d2b288b7d55c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:53.519930 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca08d56-e0b8-4021-b961-8dff4c96012b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:53.520735 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca08d56-e0b8-4021-b961-8dff4c96012b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:53.521979 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca08d56-e0b8-4021-b961-8dff4c96012b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:53.522275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca08d56-e0b8-4021-b961-8dff4c96012b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:53.530161 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:53.534677 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca08d56-e0b8-4021-b961-8dff4c96012b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:53.542436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-4ca08d56-e0b8-4021-b961-8dff4c96012b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:53.542436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:53.542436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:53.542436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:53.542436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:53.542436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:53.542436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:53.542436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:53.542436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:53.542436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:53.542436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:53.542436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:53.542436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:53.542436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:53.542436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:53.542436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:53.544073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:53.544073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:53.544073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:53.544073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:53.544073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:53.544073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:53.544073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:53.544073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:53.544073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:53.544073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:53.544073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:53.544073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:53.544073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:53.544073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:53.544073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:53.544073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:53.544919 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:53.544919 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:53.544919 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:53.544919 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:53.544919 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:53.544919 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:53.544919 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:53.544919 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:53.544919 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:53.544919 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:53.544919 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:13:53.544919 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:53.544919 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 335/679] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 30 16:13:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:53.544919 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca08d56-e0b8-4021-b961-8dff4c96012b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:53.553197 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f6e85-5de2-4502-aa29-291b5df593a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:53.553986 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f6e85-5de2-4502-aa29-291b5df593a7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:53.554275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f6e85-5de2-4502-aa29-291b5df593a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:53.554555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f6e85-5de2-4502-aa29-291b5df593a7 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:53.555375 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f6e85-5de2-4502-aa29-291b5df593a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:53.555725 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f6e85-5de2-4502-aa29-291b5df593a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:53.557212 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f6e85-5de2-4502-aa29-291b5df593a7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:53.566100 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:53.571048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 345/680] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:13:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:53.571842 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f6e85-5de2-4502-aa29-291b5df593a7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:53.664148 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca08d56-e0b8-4021-b961-8dff4c96012b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:53.686455 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f6e85-5de2-4502-aa29-291b5df593a7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:53.701316 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2f87c-975f-4de0-8b92-1dab2fb9bf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:53.701998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2f87c-975f-4de0-8b92-1dab2fb9bf24 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:53.702283 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2f87c-975f-4de0-8b92-1dab2fb9bf24 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:53.702557 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa2f87c-975f-4de0-8b92-1dab2fb9bf24 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:53.703335 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa2f87c-975f-4de0-8b92-1dab2fb9bf24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:53.703701 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa2f87c-975f-4de0-8b92-1dab2fb9bf24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:53.705096 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa2f87c-975f-4de0-8b92-1dab2fb9bf24 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:53.723857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 336/681] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:13:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:53.723857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2f87c-975f-4de0-8b92-1dab2fb9bf24 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:53.730994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdbc94-f914-43bb-8506-6418e35b9b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:53.731927 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdbc94-f914-43bb-8506-6418e35b9b98 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:53.732214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdbc94-f914-43bb-8506-6418e35b9b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:53.732492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdbc94-f914-43bb-8506-6418e35b9b98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:53.746682 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fdbc94-f914-43bb-8506-6418e35b9b98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:53.748608 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fdbc94-f914-43bb-8506-6418e35b9b98 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:53.767697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-e6fdbc94-f914-43bb-8506-6418e35b9b98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:53.767697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:53.767697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:53.767697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:53.767697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:53.767697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:53.767697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:53.767697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:53.767697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:53.767697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:53.767697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:53.767697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:53.767697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:53.767697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:53.767697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:53.768610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:53.768610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:53.768610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:53.768610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:53.768610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:53.768610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:53.768610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:53.768610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:53.768610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:53.768610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:53.768610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:53.768610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:53.768610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:53.768610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:53.768610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:53.768610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:53.768610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:53.769521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:53.769521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:53.769521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:53.769521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:53.774859 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 346/682] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:53.775599 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdbc94-f914-43bb-8506-6418e35b9b98 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:53.806504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:53.814935 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:53.844698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa2f87c-975f-4de0-8b92-1dab2fb9bf24 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:53.882274 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a461e-d469-42fb-b93f-62b9caf86303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:53.882688 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdbc94-f914-43bb-8506-6418e35b9b98 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:53.882947 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a461e-d469-42fb-b93f-62b9caf86303 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:53.883232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a461e-d469-42fb-b93f-62b9caf86303 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:53.883533 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948a461e-d469-42fb-b93f-62b9caf86303 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:53.884556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a461e-d469-42fb-b93f-62b9caf86303 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:53.884869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a461e-d469-42fb-b93f-62b9caf86303 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:53.887102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948a461e-d469-42fb-b93f-62b9caf86303 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:53.898363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 337/683] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 30 16:13:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:53.899959 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a461e-d469-42fb-b93f-62b9caf86303 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:53.923252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce69e0-df3f-46dc-95bc-6df36a9d3b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:53.923849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce69e0-df3f-46dc-95bc-6df36a9d3b27 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:53.924129 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce69e0-df3f-46dc-95bc-6df36a9d3b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:53.924404 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce69e0-df3f-46dc-95bc-6df36a9d3b27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:53.925231 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce69e0-df3f-46dc-95bc-6df36a9d3b27 None admin] RBAC: Adding query filter params (role.id=57bc5df632b3407d97116ae34d92a5cb, scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d, user.id=ec41c84c9375437590c42621ca5b400a) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:53.925546 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce69e0-df3f-46dc-95bc-6df36a9d3b27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:53.927017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce69e0-df3f-46dc-95bc-6df36a9d3b27 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:53.933413 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 347/684] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:13:53 2022] GET /identity/v3/role_assignments?user.id=ec41c84c9375437590c42621ca5b400a&scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d&role.id=57bc5df632b3407d97116ae34d92a5cb => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:53.934194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce69e0-df3f-46dc-95bc-6df36a9d3b27 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:54.054177 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:54.062261 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948a461e-d469-42fb-b93f-62b9caf86303 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:54.068555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce69e0-df3f-46dc-95bc-6df36a9d3b27 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:54.110996 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9917418-8fee-4698-834a-d308290c096e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:54.111929 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9917418-8fee-4698-834a-d308290c096e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:54.112390 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9917418-8fee-4698-834a-d308290c096e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:54.112716 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9917418-8fee-4698-834a-d308290c096e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:54.113647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9917418-8fee-4698-834a-d308290c096e None admin] RBAC: Adding query filter params (role.id=57bc5df632b3407d97116ae34d92a5cb, scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d, user.id=9b508dd8b7244d5f979e1ce7787d2c67) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:54.113897 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9917418-8fee-4698-834a-d308290c096e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:54.114357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b8627-d44e-4d6d-a772-b88beb7d4572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:54.115053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b8627-d44e-4d6d-a772-b88beb7d4572 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:54.115341 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b8627-d44e-4d6d-a772-b88beb7d4572 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:54.115580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9917418-8fee-4698-834a-d308290c096e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:54.115717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b8627-d44e-4d6d-a772-b88beb7d4572 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:54.116394 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b8627-d44e-4d6d-a772-b88beb7d4572 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:54.116704 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b8627-d44e-4d6d-a772-b88beb7d4572 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:54.118080 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b8627-d44e-4d6d-a772-b88beb7d4572 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:54.122246 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 338/685] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:13:53 2022] GET /identity/v3/role_assignments?user.id=9b508dd8b7244d5f979e1ce7787d2c67&scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d&role.id=57bc5df632b3407d97116ae34d92a5cb => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:54.123072 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9917418-8fee-4698-834a-d308290c096e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:54.125786 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:54.135086 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 348/686] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:54.144547 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b8627-d44e-4d6d-a772-b88beb7d4572 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:54.277007 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9917418-8fee-4698-834a-d308290c096e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:54.280527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b8627-d44e-4d6d-a772-b88beb7d4572 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:54.330572 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa470d0-dbb8-4ad3-9b17-e97ad272c111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:54.331112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a0f61-048e-4619-88fa-a951810adf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:54.331381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa470d0-dbb8-4ad3-9b17-e97ad272c111 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:54.331714 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa470d0-dbb8-4ad3-9b17-e97ad272c111 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:54.331911 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a0f61-048e-4619-88fa-a951810adf05 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:54.332005 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa470d0-dbb8-4ad3-9b17-e97ad272c111 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:54.332175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a0f61-048e-4619-88fa-a951810adf05 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:54.332519 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a0f61-048e-4619-88fa-a951810adf05 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:54.344128 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a0f61-048e-4619-88fa-a951810adf05 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:54.345816 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa470d0-dbb8-4ad3-9b17-e97ad272c111 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:54.345933 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a0f61-048e-4619-88fa-a951810adf05 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:54.347245 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa470d0-dbb8-4ad3-9b17-e97ad272c111 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:54.363453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-fd0a0f61-048e-4619-88fa-a951810adf05 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 16:13:54.363453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:54.363453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:54.363453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:54.363453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:54.363453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:54.363453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:54.363453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:54.363453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:54.363453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:54.363453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:54.363453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:54.363453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:13:54.363453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:13:54.363453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:13:54.363453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:13:54.364446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:54.364446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:54.364446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:13:54.364446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:54.364446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:13:54.364446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:54.364446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:54.364446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:54.364446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:54.364446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:54.364446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:54.364446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:54.364446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:54.364446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:54.364446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:54.364446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:54.364446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:54.364446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:54.365438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:54.365438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:54.365438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:13:54.365438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:13:54.365438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:13:54.365438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:13:54.365438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:13:54.365438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:13:54.365438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 16:13:54.365438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:54.369421 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 349/687] 10.222.0.67 () {56 vars in 1093 bytes} [Thu Jun 30 16:13:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:54.369970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-7aa470d0-dbb8-4ad3-9b17-e97ad272c111 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:54.369970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:54.369970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:54.369970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:54.369970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:54.369970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:54.369970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:54.369970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:54.369970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:54.369970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:54.369970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:54.369970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:54.369970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:54.369970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:54.369970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:54.370796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:54.370796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:54.370796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:54.370796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:54.370796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:54.370796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:54.370796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:54.370796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:54.370796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:54.370796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:54.370796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:54.370796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:54.370796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:54.370796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:54.370796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:54.370796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:54.370796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:54.371729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:54.371729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:54.371729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:54.371729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:54.371945 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a0f61-048e-4619-88fa-a951810adf05 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:54.372029 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 339/688] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:54.377359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa470d0-dbb8-4ad3-9b17-e97ad272c111 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:54.388839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:54.453467 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:54.504589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a0f61-048e-4619-88fa-a951810adf05 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:54.512269 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa470d0-dbb8-4ad3-9b17-e97ad272c111 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:54.543009 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f8118-1dcf-4de3-bcd9-382386624ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:54.543698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f8118-1dcf-4de3-bcd9-382386624ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:54.544087 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f8118-1dcf-4de3-bcd9-382386624ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:54.544487 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40f8118-1dcf-4de3-bcd9-382386624ae8 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:54.545540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f8118-1dcf-4de3-bcd9-382386624ae8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:54.546184 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f8118-1dcf-4de3-bcd9-382386624ae8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:54.547696 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40f8118-1dcf-4de3-bcd9-382386624ae8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:54.551281 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4029932-b0d6-4b75-8652-7ea52ac5c9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:54.552130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4029932-b0d6-4b75-8652-7ea52ac5c9d1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:54.553089 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4029932-b0d6-4b75-8652-7ea52ac5c9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:54.553779 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4029932-b0d6-4b75-8652-7ea52ac5c9d1 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:54.554714 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4029932-b0d6-4b75-8652-7ea52ac5c9d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:54.555167 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4029932-b0d6-4b75-8652-7ea52ac5c9d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:54.556703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4029932-b0d6-4b75-8652-7ea52ac5c9d1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:54.571617 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 340/690] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:54.571826 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 350/690] 10.222.0.67 () {56 vars in 1133 bytes} [Thu Jun 30 16:13:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:54.574750 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f8118-1dcf-4de3-bcd9-382386624ae8 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:54.576395 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4029932-b0d6-4b75-8652-7ea52ac5c9d1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:54.654102 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:54.686455 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4029932-b0d6-4b75-8652-7ea52ac5c9d1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:54.687694 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40f8118-1dcf-4de3-bcd9-382386624ae8 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:54.704003 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:54.725708 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd801b-7dfd-4b94-9abd-bf81e0eaa426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:54.726847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd801b-7dfd-4b94-9abd-bf81e0eaa426 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:54.727220 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd801b-7dfd-4b94-9abd-bf81e0eaa426 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:54.727620 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd801b-7dfd-4b94-9abd-bf81e0eaa426 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:54.728345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de281654-f2b2-4f54-b9f1-4bb4b0868937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:54.729173 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de281654-f2b2-4f54-b9f1-4bb4b0868937 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:54.729639 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de281654-f2b2-4f54-b9f1-4bb4b0868937 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:54.730108 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de281654-f2b2-4f54-b9f1-4bb4b0868937 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:54.736253 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcd801b-7dfd-4b94-9abd-bf81e0eaa426 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:54.737837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcd801b-7dfd-4b94-9abd-bf81e0eaa426 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:54.754689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-5fcd801b-7dfd-4b94-9abd-bf81e0eaa426 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 16:13:54.754689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:54.754689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:54.754689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:54.754689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:54.754689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:54.754689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:54.754689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:54.754689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:54.754689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:54.754689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:54.754689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:54.754689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:13:54.754689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:13:54.754689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:13:54.754689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:13:54.756580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:54.756580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:54.756580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:13:54.756580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:54.756580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:13:54.756580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:13:54.756580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:54.756580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:54.756580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:54.756580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:54.756580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:54.756580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:54.756580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:54.756580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:54.756580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:54.756580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:54.756580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:54.756580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:54.757744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:54.757744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:54.757744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:13:54.757744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:13:54.757744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:13:54.757744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:13:54.757744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:13:54.757744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:13:54.757744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 16:13:54.757744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:54.760899 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 341/691] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:13:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:54.772790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd801b-7dfd-4b94-9abd-bf81e0eaa426 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:54.819340 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de281654-f2b2-4f54-b9f1-4bb4b0868937 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:54.821247 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de281654-f2b2-4f54-b9f1-4bb4b0868937 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:54.897753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:54.939307 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 351/692] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:13:54.940940 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de281654-f2b2-4f54-b9f1-4bb4b0868937 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:54.997525 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd801b-7dfd-4b94-9abd-bf81e0eaa426 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:55.071152 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de281654-f2b2-4f54-b9f1-4bb4b0868937 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:55.080261 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8a548-17a5-40bd-9950-b91fbaa0b871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:55.081617 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8a548-17a5-40bd-9950-b91fbaa0b871 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:55.082111 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8a548-17a5-40bd-9950-b91fbaa0b871 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:55.082613 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8a548-17a5-40bd-9950-b91fbaa0b871 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:55.083744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa8a548-17a5-40bd-9950-b91fbaa0b871 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:55.084812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa8a548-17a5-40bd-9950-b91fbaa0b871 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:55.086697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa8a548-17a5-40bd-9950-b91fbaa0b871 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:55.112837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 342/693] 10.222.0.67 () {56 vars in 1135 bytes} [Thu Jun 30 16:13:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:55.118144 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8a548-17a5-40bd-9950-b91fbaa0b871 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:55.159376 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:55.160240 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26da80c-26c5-40ff-9e2e-cd2d73733a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:55.160913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26da80c-26c5-40ff-9e2e-cd2d73733a30 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:55.161237 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26da80c-26c5-40ff-9e2e-cd2d73733a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:55.161553 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26da80c-26c5-40ff-9e2e-cd2d73733a30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:55.186566 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26da80c-26c5-40ff-9e2e-cd2d73733a30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:55.188113 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26da80c-26c5-40ff-9e2e-cd2d73733a30 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:55.226431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-c26da80c-26c5-40ff-9e2e-cd2d73733a30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:55.226431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:55.226431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:55.226431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:55.226431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:55.226431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:55.226431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:55.226431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:55.226431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:55.226431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:55.226431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:55.226431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:55.226431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:55.226431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:55.226431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:55.227447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:55.227447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:55.227447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:55.227447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:55.227447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:55.227447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:55.227447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:55.227447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:55.227447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:55.227447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:55.227447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:55.227447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:55.227447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:55.227447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:55.227447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:55.227447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:55.227447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:55.229021 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:55.229021 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:55.229021 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:55.229021 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:55.229021 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 352/694] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:13:55.233214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26da80c-26c5-40ff-9e2e-cd2d73733a30 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:55.260101 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:55.294675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8a548-17a5-40bd-9950-b91fbaa0b871 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:55.359887 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc187b6-a659-4f6d-b0f1-2428adf9ce47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:55.360461 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc187b6-a659-4f6d-b0f1-2428adf9ce47 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:55.360749 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc187b6-a659-4f6d-b0f1-2428adf9ce47 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:55.361018 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc187b6-a659-4f6d-b0f1-2428adf9ce47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:55.392301 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26da80c-26c5-40ff-9e2e-cd2d73733a30 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:55.436341 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6833db89-8334-45d6-89f9-5a85f05d3d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:55.437098 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833db89-8334-45d6-89f9-5a85f05d3d5c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:55.437417 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833db89-8334-45d6-89f9-5a85f05d3d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:55.437723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833db89-8334-45d6-89f9-5a85f05d3d5c None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:55.438493 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6833db89-8334-45d6-89f9-5a85f05d3d5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:55.438832 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6833db89-8334-45d6-89f9-5a85f05d3d5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:55.440512 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6833db89-8334-45d6-89f9-5a85f05d3d5c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:55.454331 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 353/695] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:55.460259 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab2fd2-870d-4969-b976-d171414e9013 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:55.460611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab2fd2-870d-4969-b976-d171414e9013 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:55.460850 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab2fd2-870d-4969-b976-d171414e9013 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:55.466204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 354/696] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:55.466204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab2fd2-870d-4969-b976-d171414e9013 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:55.485101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc187b6-a659-4f6d-b0f1-2428adf9ce47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:55.486803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc187b6-a659-4f6d-b0f1-2428adf9ce47 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:55.522781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:55.580554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:55.598192 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 343/697] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:13:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 16:13:55.600116 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca431686-e3fc-4659-8db5-26e229d3ac93 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:55.600488 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca431686-e3fc-4659-8db5-26e229d3ac93 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:55.600860 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca431686-e3fc-4659-8db5-26e229d3ac93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:55.608127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab2fd2-870d-4969-b976-d171414e9013 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:55.620493 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ca431686-e3fc-4659-8db5-26e229d3ac93 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:55.621753 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ca431686-e3fc-4659-8db5-26e229d3ac93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:55.658733 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbcf8f-c339-43f9-9de3-67c9d937dd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:55.660803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbcf8f-c339-43f9-9de3-67c9d937dd28 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:55.661327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbcf8f-c339-43f9-9de3-67c9d937dd28 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:55.662643 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfbcf8f-c339-43f9-9de3-67c9d937dd28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:55.671915 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfbcf8f-c339-43f9-9de3-67c9d937dd28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:55.674036 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfbcf8f-c339-43f9-9de3-67c9d937dd28 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:55.694535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-7bfbcf8f-c339-43f9-9de3-67c9d937dd28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:55.694535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:55.694535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:55.694535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:55.694535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:55.694535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:55.694535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:55.694535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:55.694535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:55.694535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:55.694535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:55.694535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:55.694535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:13:55.694535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:13:55.694535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:13:55.695461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:13:55.695461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:55.695461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:55.695461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:55.695461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:55.695461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:55.695461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:55.695461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:55.695461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:55.695461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:55.695461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:55.695461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:55.695461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:55.695461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:55.695461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:55.695461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:55.695461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:55.696358 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:13:55.696358 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:13:55.696358 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:13:55.696358 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:13:55.696358 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:13:55.696358 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:13:55.696358 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:55.696358 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:55.699814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 355/698] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:13:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:55.700832 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbcf8f-c339-43f9-9de3-67c9d937dd28 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:55.708207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 344/699] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:55.708207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca431686-e3fc-4659-8db5-26e229d3ac93 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:55.791666 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:55.843032 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfbcf8f-c339-43f9-9de3-67c9d937dd28 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:55.848122 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca431686-e3fc-4659-8db5-26e229d3ac93 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:55.879796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d80666-3d6d-445c-b419-5c085d15edb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:55.881198 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d80666-3d6d-445c-b419-5c085d15edb3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:55.882035 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d80666-3d6d-445c-b419-5c085d15edb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:55.882386 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d80666-3d6d-445c-b419-5c085d15edb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:55.893508 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d80666-3d6d-445c-b419-5c085d15edb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:55.895199 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d80666-3d6d-445c-b419-5c085d15edb3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:55.895644 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d84c1-29aa-4357-b52d-d7f5de725164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:55.897093 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d84c1-29aa-4357-b52d-d7f5de725164 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:55.897562 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d84c1-29aa-4357-b52d-d7f5de725164 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:55.898019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d84c1-29aa-4357-b52d-d7f5de725164 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:55.898976 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:55.900908 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d84c1-29aa-4357-b52d-d7f5de725164 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:55.901416 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d84c1-29aa-4357-b52d-d7f5de725164 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:55.903087 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d84c1-29aa-4357-b52d-d7f5de725164 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:55.916999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-b2d80666-3d6d-445c-b419-5c085d15edb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:55.916999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:55.916999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:55.916999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:55.916999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:55.916999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:55.916999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:55.916999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:55.916999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:55.916999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:55.916999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:55.916999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:55.916999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:55.916999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:55.916999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:55.917942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:55.917942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:55.917942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:55.917942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:55.917942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:55.917942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:55.917942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:55.917942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:55.917942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:55.917942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:55.917942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:55.917942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:55.917942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:55.917942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:55.917942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:55.917942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:55.917942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:55.919089 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:55.919089 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:55.919089 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:55.919089 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:55.919089 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 356/700] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:55.920206 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d80666-3d6d-445c-b419-5c085d15edb3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:55.920518 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 345/701] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:13:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:55.923820 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d84c1-29aa-4357-b52d-d7f5de725164 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:56.059082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d80666-3d6d-445c-b419-5c085d15edb3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:56.060218 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d84c1-29aa-4357-b52d-d7f5de725164 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:56.090631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:56.108645 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc8193-d2c9-4c2d-9c34-25a043959215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:56.109312 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc8193-d2c9-4c2d-9c34-25a043959215 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:56.109589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc8193-d2c9-4c2d-9c34-25a043959215 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:56.109857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc8193-d2c9-4c2d-9c34-25a043959215 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:56.110382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d521a-e7bf-4fc7-8f7e-32f23be64065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:56.110571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc8193-d2c9-4c2d-9c34-25a043959215 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:56.110875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc8193-d2c9-4c2d-9c34-25a043959215 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:56.111698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d521a-e7bf-4fc7-8f7e-32f23be64065 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:56.112146 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d521a-e7bf-4fc7-8f7e-32f23be64065 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:56.112259 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc8193-d2c9-4c2d-9c34-25a043959215 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:56.112660 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d521a-e7bf-4fc7-8f7e-32f23be64065 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:56.113522 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d521a-e7bf-4fc7-8f7e-32f23be64065 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:56.116845 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d521a-e7bf-4fc7-8f7e-32f23be64065 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:56.127716 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 346/702] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:56.128651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc8193-d2c9-4c2d-9c34-25a043959215 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:56.139749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-b56d521a-e7bf-4fc7-8f7e-32f23be64065 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 16:13:56.139749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:56.139749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:13:56.139749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:13:56.139749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:13:56.139749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:13:56.139749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:13:56.139749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:13:56.139749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 16:13:56.139749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:56.139749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:13:56.139749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:56.139749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:56.139749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:56.139749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:56.139749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:56.140710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:56.140710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:56.140710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:56.140710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:56.140710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:56.140710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:56.140710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:56.140710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:13:56.140710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:13:56.140710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:13:56.140710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:13:56.140710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:56.140710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:56.140710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:56.140710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:56.140710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:56.141564 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:56.141564 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:56.141564 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:56.141564 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:56.141564 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:56.141564 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:56.141564 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:56.141564 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:56.141564 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:56.141564 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:56.141564 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:56.141564 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:13:56.141564 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:13:56.141564 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 16:13:56.141564 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:56.150231 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 357/703] 10.222.0.67 () {56 vars in 1129 bytes} [Thu Jun 30 16:13:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:56.150231 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d521a-e7bf-4fc7-8f7e-32f23be64065 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:56.228867 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:56.260652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc8193-d2c9-4c2d-9c34-25a043959215 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:56.290037 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d521a-e7bf-4fc7-8f7e-32f23be64065 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:56.299428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb953f4-f0bd-4508-a592-fd089bddaf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:56.300200 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb953f4-f0bd-4508-a592-fd089bddaf5a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:56.300547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb953f4-f0bd-4508-a592-fd089bddaf5a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:56.300891 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb953f4-f0bd-4508-a592-fd089bddaf5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:56.301879 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb953f4-f0bd-4508-a592-fd089bddaf5a None admin] RBAC: Adding query filter params (role.id=8acc34a163874a60b03728b814eaf075, scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d, user.id=71cca3708386459197abf6c67505bc5f) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:56.302260 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb953f4-f0bd-4508-a592-fd089bddaf5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:56.304486 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb953f4-f0bd-4508-a592-fd089bddaf5a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:56.315845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 347/704] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:13:56 2022] GET /identity/v3/role_assignments?user.id=71cca3708386459197abf6c67505bc5f&scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d&role.id=8acc34a163874a60b03728b814eaf075 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:56.316983 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb953f4-f0bd-4508-a592-fd089bddaf5a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:56.335945 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745f58d-ae0e-4d36-b3c4-1c33d93c4f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:56.337141 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745f58d-ae0e-4d36-b3c4-1c33d93c4f03 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:56.337607 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745f58d-ae0e-4d36-b3c4-1c33d93c4f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:56.338075 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f745f58d-ae0e-4d36-b3c4-1c33d93c4f03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:56.347865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f745f58d-ae0e-4d36-b3c4-1c33d93c4f03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:56.350189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f745f58d-ae0e-4d36-b3c4-1c33d93c4f03 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:56.369505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f745f58d-ae0e-4d36-b3c4-1c33d93c4f03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:56.369505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:56.369505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:56.369505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:56.369505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:56.369505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:56.369505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:56.369505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:56.369505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:56.369505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:56.369505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:56.369505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:56.369505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:13:56.369505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:13:56.369505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:13:56.370497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:13:56.370497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:56.370497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:56.370497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:56.370497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:56.370497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:56.370497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:56.370497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:56.370497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:56.370497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:56.370497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:56.370497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:56.370497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:56.370497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:56.370497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:56.370497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:56.370497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:56.371377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:13:56.371377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:13:56.371377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:13:56.371377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:13:56.371377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:13:56.371377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:13:56.371377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:13:56.371377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:56.374468 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 358/705] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:13:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:56.380472 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745f58d-ae0e-4d36-b3c4-1c33d93c4f03 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:56.411285 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:56.501471 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb953f4-f0bd-4508-a592-fd089bddaf5a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:56.508528 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f745f58d-ae0e-4d36-b3c4-1c33d93c4f03 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:56.538915 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:56.553302 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7dc40e-2267-4e78-81c0-d7e998d98d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:56.553719 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb2cf09-ba99-4b8b-a2ef-7d4073d65ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:56.553967 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7dc40e-2267-4e78-81c0-d7e998d98d01 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:56.554251 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7dc40e-2267-4e78-81c0-d7e998d98d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:56.554527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7dc40e-2267-4e78-81c0-d7e998d98d01 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:56.555024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2cf09-ba99-4b8b-a2ef-7d4073d65ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:56.555313 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7dc40e-2267-4e78-81c0-d7e998d98d01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:56.555657 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7dc40e-2267-4e78-81c0-d7e998d98d01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:56.555768 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2cf09-ba99-4b8b-a2ef-7d4073d65ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:56.556185 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2cf09-ba99-4b8b-a2ef-7d4073d65ed6 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:56.557035 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb2cf09-ba99-4b8b-a2ef-7d4073d65ed6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:56.557148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7dc40e-2267-4e78-81c0-d7e998d98d01 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:56.557578 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb2cf09-ba99-4b8b-a2ef-7d4073d65ed6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:56.559343 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb2cf09-ba99-4b8b-a2ef-7d4073d65ed6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:56.570680 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 348/706] 10.222.0.67 () {56 vars in 1124 bytes} [Thu Jun 30 16:13:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:56.570680 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6334e5a-7755-4d11-a6af-dcb9bd490dae None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:56.570680 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6334e5a-7755-4d11-a6af-dcb9bd490dae None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:56.570680 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6334e5a-7755-4d11-a6af-dcb9bd490dae None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:56.572272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 349/707] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:56.573613 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6334e5a-7755-4d11-a6af-dcb9bd490dae None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:56.576344 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 359/708] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:13:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:56.580820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b9456-c98f-49ef-87ca-e3671db7c69f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:56.581452 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b9456-c98f-49ef-87ca-e3671db7c69f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:56.582793 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b9456-c98f-49ef-87ca-e3671db7c69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:56.606441 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-908b9456-c98f-49ef-87ca-e3671db7c69f None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:56.608470 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-908b9456-c98f-49ef-87ca-e3671db7c69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:56.694840 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 360/709] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:56.696412 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b9456-c98f-49ef-87ca-e3671db7c69f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:56.696523 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6334e5a-7755-4d11-a6af-dcb9bd490dae None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:56.718415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:56.734760 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f7834a-b02e-4758-af28-f509f3c52bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:56.735599 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7834a-b02e-4758-af28-f509f3c52bfa None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:56.735996 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7834a-b02e-4758-af28-f509f3c52bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:56.736376 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7834a-b02e-4758-af28-f509f3c52bfa None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:56.737155 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f7834a-b02e-4758-af28-f509f3c52bfa None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:56.737559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f7834a-b02e-4758-af28-f509f3c52bfa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:56.739014 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f7834a-b02e-4758-af28-f509f3c52bfa None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:56.745758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 350/710] 10.222.0.67 () {56 vars in 1124 bytes} [Thu Jun 30 16:13:56 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:56.746874 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6feec5-5b1b-40ff-877b-407cecdb6d00 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:56.747167 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6feec5-5b1b-40ff-877b-407cecdb6d00 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:56.747431 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6feec5-5b1b-40ff-877b-407cecdb6d00 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:56.752689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 351/711] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:56.752689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6feec5-5b1b-40ff-877b-407cecdb6d00 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:56.842896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b9456-c98f-49ef-87ca-e3671db7c69f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:56.856871 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:56.886386 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10f2c5-68fa-43d0-9087-e1c7501734ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:56.887304 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10f2c5-68fa-43d0-9087-e1c7501734ec None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:56.887746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10f2c5-68fa-43d0-9087-e1c7501734ec None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:56.888128 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10f2c5-68fa-43d0-9087-e1c7501734ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:56.889320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10f2c5-68fa-43d0-9087-e1c7501734ec None admin] RBAC: Adding query filter params (role.id=57bc5df632b3407d97116ae34d92a5cb, scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d, user.id=988b917eea4b48ffae32e23b8555a5c6) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:56.889643 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10f2c5-68fa-43d0-9087-e1c7501734ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:56.891562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10f2c5-68fa-43d0-9087-e1c7501734ec None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:56.900168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 361/712] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:13:56 2022] GET /identity/v3/role_assignments?user.id=988b917eea4b48ffae32e23b8555a5c6&scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d&role.id=57bc5df632b3407d97116ae34d92a5cb => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:56.901755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106adec-40b3-4805-947d-be378de1cb38 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:56.902165 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106adec-40b3-4805-947d-be378de1cb38 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:56.902557 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a106adec-40b3-4805-947d-be378de1cb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:56.905130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6feec5-5b1b-40ff-877b-407cecdb6d00 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:56.922352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a106adec-40b3-4805-947d-be378de1cb38 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:56.924716 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a106adec-40b3-4805-947d-be378de1cb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:56.954656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e497f-3641-4c99-8f89-d99fb0ce861e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:56.955384 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e497f-3641-4c99-8f89-d99fb0ce861e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:56.955702 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e497f-3641-4c99-8f89-d99fb0ce861e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:56.955977 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e497f-3641-4c99-8f89-d99fb0ce861e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:56.971977 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e497f-3641-4c99-8f89-d99fb0ce861e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:56.974109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e497f-3641-4c99-8f89-d99fb0ce861e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:56.997650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-194e497f-3641-4c99-8f89-d99fb0ce861e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:56.997650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:56.997650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:56.997650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:56.997650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:56.997650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:56.997650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:56.997650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:56.997650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:56.997650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:56.997650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:56.997650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:56.997650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:56.997650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:56.997650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:56.998656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:56.998656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:56.998656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:56.998656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:56.998656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:56.998656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:56.998656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:56.998656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:56.998656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:56.998656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:56.998656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:56.998656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:56.998656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:56.998656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:56.998656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:56.998656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:56.998656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:57.000083 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:57.000083 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:57.000083 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:57.000083 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:57.006916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 352/713] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:57.006916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e497f-3641-4c99-8f89-d99fb0ce861e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:57.056788 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:57.060848 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 362/714] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:57.106291 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106adec-40b3-4805-947d-be378de1cb38 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:57.174577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e497f-3641-4c99-8f89-d99fb0ce861e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:57.214317 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd9eb9-8177-4056-b7ec-3fed7d851895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:57.215348 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd9eb9-8177-4056-b7ec-3fed7d851895 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:57.215862 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd9eb9-8177-4056-b7ec-3fed7d851895 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:57.216341 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd9eb9-8177-4056-b7ec-3fed7d851895 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:57.217273 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cd9eb9-8177-4056-b7ec-3fed7d851895 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:57.217785 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cd9eb9-8177-4056-b7ec-3fed7d851895 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:57.219458 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cd9eb9-8177-4056-b7ec-3fed7d851895 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:57.232916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 353/715] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:57.238739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd9eb9-8177-4056-b7ec-3fed7d851895 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:57.246984 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a106adec-40b3-4805-947d-be378de1cb38 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:57.248970 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:57.287146 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe84c0b-2112-4473-a07c-da6896345a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:57.288340 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe84c0b-2112-4473-a07c-da6896345a8c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:57.288856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe84c0b-2112-4473-a07c-da6896345a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:57.289343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe84c0b-2112-4473-a07c-da6896345a8c None admin] PATH_INFO: `/v3/services/network` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:57.290209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe84c0b-2112-4473-a07c-da6896345a8c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:57.292624 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe84c0b-2112-4473-a07c-da6896345a8c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:57.306835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-9fe84c0b-2112-4473-a07c-da6896345a8c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 16:13:57.306835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:57.306835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:13:57.306835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:13:57.306835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:13:57.306835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:13:57.306835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:13:57.306835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:13:57.306835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 16:13:57.306835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:57.306835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:13:57.306835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:57.306835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:57.306835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:57.306835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:57.306835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:57.306835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:57.307900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:57.307900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:57.307900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:57.307900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:57.307900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:57.307900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:57.307900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:13:57.307900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:13:57.307900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:13:57.307900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:13:57.307900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:57.307900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:57.307900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:57.307900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:57.307900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:57.307900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:57.308723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:57.308723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:57.308723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:57.308723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:57.308723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:57.308723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:57.308723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:57.308723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:57.308723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:57.308723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:57.308723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:13:57.308723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:13:57.308723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 16:13:57.308723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:13:57.311716 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 363/716] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:13:57 2022] GET /identity/v3/services/network => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:57.317654 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe84c0b-2112-4473-a07c-da6896345a8c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:57.345366 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:57.364873 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd9eb9-8177-4056-b7ec-3fed7d851895 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:57.409892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede20f4-24bc-40e1-b124-5d0cb334003a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:57.410938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede20f4-24bc-40e1-b124-5d0cb334003a None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:57.411223 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede20f4-24bc-40e1-b124-5d0cb334003a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:57.411534 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede20f4-24bc-40e1-b124-5d0cb334003a None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:57.412740 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede20f4-24bc-40e1-b124-5d0cb334003a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:57.414473 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede20f4-24bc-40e1-b124-5d0cb334003a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:57.437079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe84c0b-2112-4473-a07c-da6896345a8c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:57.475188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef9216-2b4c-45ed-9870-63ad29dada98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:57.475632 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef9216-2b4c-45ed-9870-63ad29dada98 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:57.476021 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef9216-2b4c-45ed-9870-63ad29dada98 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:57.476291 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef9216-2b4c-45ed-9870-63ad29dada98 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:57.476997 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef9216-2b4c-45ed-9870-63ad29dada98 None admin] RBAC: Adding query filter params (name=network) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:57.477304 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef9216-2b4c-45ed-9870-63ad29dada98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:57.478807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef9216-2b4c-45ed-9870-63ad29dada98 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:57.483824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 364/717] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 30 16:13:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:13:57.489066 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef9216-2b4c-45ed-9870-63ad29dada98 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:57.522342 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:57.562227 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 354/718] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 30 16:13:57 2022] POST /identity/v3/users => generated 254 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:13:57.600092 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef9216-2b4c-45ed-9870-63ad29dada98 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:57.654037 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e98468-f34e-4190-be63-55f8f6f881a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:57.654737 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e98468-f34e-4190-be63-55f8f6f881a9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:57.655020 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e98468-f34e-4190-be63-55f8f6f881a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:57.655295 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e98468-f34e-4190-be63-55f8f6f881a9 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:57.656299 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e98468-f34e-4190-be63-55f8f6f881a9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:57.656611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e98468-f34e-4190-be63-55f8f6f881a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:57.658095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e98468-f34e-4190-be63-55f8f6f881a9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:57.669943 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 365/719] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 30 16:13:57 2022] GET /identity/v3/services?type=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:58.792632 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509e400-2a24-4910-9766-aaa7e4c04f7b None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:58.793159 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509e400-2a24-4910-9766-aaa7e4c04f7b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:58.793606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509e400-2a24-4910-9766-aaa7e4c04f7b None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:58.796379 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 355/720] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:58.804828 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45991d46-af95-435c-a739-aebd157aec22 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:58.805290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45991d46-af95-435c-a739-aebd157aec22 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:58.805481 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45991d46-af95-435c-a739-aebd157aec22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:58.828427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-45991d46-af95-435c-a739-aebd157aec22 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:58.829636 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-45991d46-af95-435c-a739-aebd157aec22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:58.922345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 366/721] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:58.944881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509e400-2a24-4910-9766-aaa7e4c04f7b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:59.178394 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509e400-2a24-4910-9766-aaa7e4c04f7b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:59.231892 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f003bc-ae76-4f76-ae50-1b5c7ead2109 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:59.232562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f003bc-ae76-4f76-ae50-1b5c7ead2109 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:59.233041 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f003bc-ae76-4f76-ae50-1b5c7ead2109 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:59.235220 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 367/722] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:13:59.241560 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323c703-1d11-4dac-8085-6d24bda09eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:59.242486 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323c703-1d11-4dac-8085-6d24bda09eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:59.242773 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323c703-1d11-4dac-8085-6d24bda09eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:59.243048 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323c703-1d11-4dac-8085-6d24bda09eb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:59.244368 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be34f1-ac99-4bb1-a082-cc3a3490d073 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:59.244876 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be34f1-ac99-4bb1-a082-cc3a3490d073 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:59.245365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be34f1-ac99-4bb1-a082-cc3a3490d073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:59.263907 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c323c703-1d11-4dac-8085-6d24bda09eb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:13:59.277445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-c323c703-1d11-4dac-8085-6d24bda09eb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:13:59.277445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:59.277445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:59.277445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:59.277445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:59.277445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:59.277445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:59.277445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:59.277445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:59.277445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:59.277445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:59.277445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:59.277445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:13:59.277445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:13:59.277445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:13:59.277445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:13:59.278464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:13:59.278464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:13:59.278464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:13:59.278464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:13:59.278464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:59.278464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:59.278464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:59.278464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:59.278464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:59.278464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:59.278464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:59.278464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:59.278464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:59.278464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:59.278464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:59.278464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:59.279400 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:59.279400 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:59.279400 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:59.279400 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:59.279400 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:13:59.279400 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:13:59.279400 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:13:59.279400 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:13:59.279400 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:13:59.279400 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:13:59.279400 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:13:59.279400 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:59.280568 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 356/723] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:13:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:13:59.286326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323c703-1d11-4dac-8085-6d24bda09eb8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:59.291896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b9be34f1-ac99-4bb1-a082-cc3a3490d073 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:59.294263 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b9be34f1-ac99-4bb1-a082-cc3a3490d073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:59.300293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:59.330830 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:59.398304 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 368/724] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:59.460876 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be34f1-ac99-4bb1-a082-cc3a3490d073 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:59.467491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323c703-1d11-4dac-8085-6d24bda09eb8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:59.501962 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab6be3-d489-4b73-ac1c-38400f57cdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:59.502992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab6be3-d489-4b73-ac1c-38400f57cdc0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:59.503426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab6be3-d489-4b73-ac1c-38400f57cdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:59.503869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab6be3-d489-4b73-ac1c-38400f57cdc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:59.504773 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab6be3-d489-4b73-ac1c-38400f57cdc0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:13:59.505265 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab6be3-d489-4b73-ac1c-38400f57cdc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:59.507519 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab6be3-d489-4b73-ac1c-38400f57cdc0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:59.516690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 357/725] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:13:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:13:59.523036 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab6be3-d489-4b73-ac1c-38400f57cdc0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:59.639834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:59.659095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be34f1-ac99-4bb1-a082-cc3a3490d073 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:59.683783 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab6be3-d489-4b73-ac1c-38400f57cdc0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:59.704757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc9ee5-2b76-4360-9b7a-b0b34d1ea5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:59.705418 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc9ee5-2b76-4360-9b7a-b0b34d1ea5af None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:59.705895 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc9ee5-2b76-4360-9b7a-b0b34d1ea5af None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:59.706336 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc9ee5-2b76-4360-9b7a-b0b34d1ea5af None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:59.707698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc9ee5-2b76-4360-9b7a-b0b34d1ea5af None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:59.709575 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc9ee5-2b76-4360-9b7a-b0b34d1ea5af None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:59.721733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b0a5cd-72dc-4372-be7f-ab0aea0a8e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:13:59.722539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0a5cd-72dc-4372-be7f-ab0aea0a8e79 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:59.722898 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0a5cd-72dc-4372-be7f-ab0aea0a8e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:59.723246 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0a5cd-72dc-4372-be7f-ab0aea0a8e79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:59.735446 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b0a5cd-72dc-4372-be7f-ab0aea0a8e79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:13:59.737087 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b0a5cd-72dc-4372-be7f-ab0aea0a8e79 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:13:59.755834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-97b0a5cd-72dc-4372-be7f-ab0aea0a8e79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:59.755834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:13:59.755834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:13:59.755834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:13:59.755834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:13:59.755834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:13:59.755834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:13:59.755834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:13:59.755834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:13:59.755834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:13:59.755834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:13:59.755834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:13:59.755834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:13:59.755834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:13:59.755834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:13:59.757420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:13:59.757420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:13:59.757420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:13:59.757420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:13:59.757420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:13:59.757420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:13:59.757420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:13:59.757420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:13:59.757420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:13:59.757420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:13:59.757420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:13:59.757420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:13:59.757420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:13:59.757420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:13:59.757420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:13:59.757420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:13:59.757420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:13:59.758359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:13:59.758359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:13:59.758359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:13:59.758359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:13:59.758359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 358/726] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:13:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:13:59.763520 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473f19b-7dbc-400f-9a4b-9828df57b339 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:59.763520 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473f19b-7dbc-400f-9a4b-9828df57b339 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:59.763520 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5473f19b-7dbc-400f-9a4b-9828df57b339 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:59.764074 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 359/727] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:13:59.765196 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473f19b-7dbc-400f-9a4b-9828df57b339 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:13:59.835423 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 369/728] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 30 16:13:59 2022] POST /identity/v3/services => generated 251 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:13:59.836544 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb1eca-a842-4eb7-8449-54b7eecff855 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:13:59.836834 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb1eca-a842-4eb7-8449-54b7eecff855 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:13:59.837110 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb1eca-a842-4eb7-8449-54b7eecff855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:13:59.861157 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f0fb1eca-a842-4eb7-8449-54b7eecff855 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:13:59.862813 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f0fb1eca-a842-4eb7-8449-54b7eecff855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:13:59.878226 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:13:59.968322 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 370/729] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:13:59 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:13:59.969782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5473f19b-7dbc-400f-9a4b-9828df57b339 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:13:59.987185 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:00.011337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461bc5a-14eb-48ff-804f-b6cea1844267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:00.012138 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461bc5a-14eb-48ff-804f-b6cea1844267 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:00.012492 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461bc5a-14eb-48ff-804f-b6cea1844267 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:00.012877 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461bc5a-14eb-48ff-804f-b6cea1844267 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:00.019507 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461bc5a-14eb-48ff-804f-b6cea1844267 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:00.019507 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461bc5a-14eb-48ff-804f-b6cea1844267 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:00.019507 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9461bc5a-14eb-48ff-804f-b6cea1844267 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:00.022822 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb1eca-a842-4eb7-8449-54b7eecff855 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:00.031536 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 360/730] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:00.036634 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461bc5a-14eb-48ff-804f-b6cea1844267 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:00.151191 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461bc5a-14eb-48ff-804f-b6cea1844267 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:00.164445 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb1eca-a842-4eb7-8449-54b7eecff855 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:00.198318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb2221-89a4-4f72-a619-d8a37530fd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:00.199915 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb2221-89a4-4f72-a619-d8a37530fd96 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:00.200680 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb2221-89a4-4f72-a619-d8a37530fd96 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:00.201137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb2221-89a4-4f72-a619-d8a37530fd96 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:00.211286 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e5564-1fc9-4d42-8427-e6bb469bd162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:00.212491 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e5564-1fc9-4d42-8427-e6bb469bd162 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:00.212847 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e5564-1fc9-4d42-8427-e6bb469bd162 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:00.213057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdb2221-89a4-4f72-a619-d8a37530fd96 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:00.213206 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e5564-1fc9-4d42-8427-e6bb469bd162 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:00.215882 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdb2221-89a4-4f72-a619-d8a37530fd96 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:00.229623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e5564-1fc9-4d42-8427-e6bb469bd162 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:14:00.233859 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-ccdb2221-89a4-4f72-a619-d8a37530fd96 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 16:14:00.233859 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:00.233859 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:00.233859 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:00.233859 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:00.233859 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:00.233859 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:00.233859 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:00.233859 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:00.233859 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:00.233859 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:00.233859 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:00.233859 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:14:00.233859 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:14:00.233859 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:14:00.233859 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:14:00.234839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:00.234839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:00.234839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:14:00.234839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:00.234839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:14:00.234839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:00.234839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:00.234839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:00.234839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:00.234839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:00.234839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:00.234839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:00.234839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:00.234839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:00.234839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:00.234839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:00.234839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:00.234839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:00.235799 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:00.235799 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:00.235799 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:14:00.235799 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:14:00.235799 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:14:00.235799 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:14:00.235799 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:14:00.235799 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:14:00.235799 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 16:14:00.235799 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:00.238215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-4a1e5564-1fc9-4d42-8427-e6bb469bd162 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:14:00.238215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:00.238215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:00.238215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:00.238215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:00.238215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:00.238215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:00.238215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:00.238215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:00.238215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:00.238215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:00.238215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:00.238215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:14:00.238215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:14:00.238215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:14:00.238215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:14:00.239122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:14:00.239122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:14:00.239122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:14:00.239122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:14:00.239122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:00.239122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:00.239122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:00.239122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:00.239122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:00.239122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:00.239122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:00.239122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:00.239122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:00.239122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:00.239122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:00.239122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:00.244392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:00.244392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:00.244392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:00.244392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:00.244392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:14:00.244392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:14:00.244392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:14:00.244392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:14:00.244392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:14:00.244392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:14:00.244392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:14:00.244392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:00.244392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 371/731] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:14:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:14:00.245097 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 361/732] 10.222.0.67 () {56 vars in 1093 bytes} [Thu Jun 30 16:14:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:00.248178 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e5564-1fc9-4d42-8427-e6bb469bd162 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:00.249318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb2221-89a4-4f72-a619-d8a37530fd96 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:00.259618 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:00.311923 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:00.385558 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb2221-89a4-4f72-a619-d8a37530fd96 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:00.401880 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e5564-1fc9-4d42-8427-e6bb469bd162 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:00.429155 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa94c67-d78a-480a-81c9-37b63944f404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:00.429831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa94c67-d78a-480a-81c9-37b63944f404 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:00.430113 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa94c67-d78a-480a-81c9-37b63944f404 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:00.430388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa94c67-d78a-480a-81c9-37b63944f404 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:00.431229 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa94c67-d78a-480a-81c9-37b63944f404 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:00.431580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa94c67-d78a-480a-81c9-37b63944f404 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:00.433624 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa94c67-d78a-480a-81c9-37b63944f404 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:00.451421 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee08aa4-ab6d-4db3-8305-c9b4688e5e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:00.452115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee08aa4-ab6d-4db3-8305-c9b4688e5e8b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:00.452395 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee08aa4-ab6d-4db3-8305-c9b4688e5e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:00.452667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee08aa4-ab6d-4db3-8305-c9b4688e5e8b None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:00.453416 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee08aa4-ab6d-4db3-8305-c9b4688e5e8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:00.453720 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee08aa4-ab6d-4db3-8305-c9b4688e5e8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:00.455115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee08aa4-ab6d-4db3-8305-c9b4688e5e8b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:00.456311 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 362/733] 10.222.0.67 () {56 vars in 1133 bytes} [Thu Jun 30 16:14:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:00.457843 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534eda2a-b0be-4ec9-8790-02d92dad93cd None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:00.460370 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534eda2a-b0be-4ec9-8790-02d92dad93cd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:00.460370 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534eda2a-b0be-4ec9-8790-02d92dad93cd None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:00.460370 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 363/734] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:00.461456 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d226fba-0256-4dd9-adf9-166b985eec3d None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:00.461799 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d226fba-0256-4dd9-adf9-166b985eec3d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:00.462080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d226fba-0256-4dd9-adf9-166b985eec3d None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:00.464177 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 364/735] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:00.464924 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d226fba-0256-4dd9-adf9-166b985eec3d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:00.469764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 372/736] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:14:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:00.471391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b2081-8bf4-4a78-8af5-0a40f60a7e81 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:00.471902 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b2081-8bf4-4a78-8af5-0a40f60a7e81 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:00.472333 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b2081-8bf4-4a78-8af5-0a40f60a7e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:00.525391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-972b2081-8bf4-4a78-8af5-0a40f60a7e81 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:00.526532 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-972b2081-8bf4-4a78-8af5-0a40f60a7e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:00.591247 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:00.606137 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 373/737] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:00 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:14:00.606137 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efed13a-383a-463d-916c-04943107d12d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:00.606137 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efed13a-383a-463d-916c-04943107d12d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:00.606137 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efed13a-383a-463d-916c-04943107d12d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:00.622243 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:00.624156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3efed13a-383a-463d-916c-04943107d12d None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:00.625364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3efed13a-383a-463d-916c-04943107d12d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:00.635446 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d226fba-0256-4dd9-adf9-166b985eec3d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:00.673417 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3e875-faaf-440f-aef0-93d9c6583df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:00.674722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3e875-faaf-440f-aef0-93d9c6583df9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:00.675010 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3e875-faaf-440f-aef0-93d9c6583df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:00.675291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f3e875-faaf-440f-aef0-93d9c6583df9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:00.686427 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f3e875-faaf-440f-aef0-93d9c6583df9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:00.687914 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f3e875-faaf-440f-aef0-93d9c6583df9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:00.694844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 374/738] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:00 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 16:14:00.695926 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efed13a-383a-463d-916c-04943107d12d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:00.707359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-97f3e875-faaf-440f-aef0-93d9c6583df9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:00.707359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:00.707359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:00.707359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:00.707359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:00.707359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:00.707359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:00.707359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:00.707359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:00.707359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:00.707359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:00.707359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:00.707359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:00.707359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:00.707359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:00.708854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:00.708854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:00.708854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:00.708854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:00.708854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:00.708854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:00.708854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:00.708854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:00.708854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:00.708854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:00.708854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:00.708854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:00.708854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:00.708854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:00.708854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:00.708854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:00.708854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:00.709755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:00.709755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:00.709755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:00.709755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:00.709755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 365/739] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:00.710246 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3e875-faaf-440f-aef0-93d9c6583df9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:00.807077 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efed13a-383a-463d-916c-04943107d12d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:00.821926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f3e875-faaf-440f-aef0-93d9c6583df9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:00.835123 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:00.844339 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8df254-8697-4cec-a0c5-1e61fabb1d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:00.845149 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8df254-8697-4cec-a0c5-1e61fabb1d2e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:00.846455 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8df254-8697-4cec-a0c5-1e61fabb1d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:00.846455 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8df254-8697-4cec-a0c5-1e61fabb1d2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:00.857005 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8df254-8697-4cec-a0c5-1e61fabb1d2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:00.858479 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8df254-8697-4cec-a0c5-1e61fabb1d2e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:00.861126 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd8625-5fd4-49ea-bc29-c7a494d611f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:00.861945 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd8625-5fd4-49ea-bc29-c7a494d611f2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:00.862232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd8625-5fd4-49ea-bc29-c7a494d611f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:00.862512 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd8625-5fd4-49ea-bc29-c7a494d611f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:00.874136 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd8625-5fd4-49ea-bc29-c7a494d611f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:14:00.876308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-da8df254-8697-4cec-a0c5-1e61fabb1d2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:00.876308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:00.876308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:00.876308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:00.876308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:00.876308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:00.876308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:00.876308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:00.876308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:00.876308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:00.876308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:00.876308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:00.876308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:00.876308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:00.876308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:00.877148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:00.877148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:00.877148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:00.877148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:00.877148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:00.877148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:00.877148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:00.877148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:00.877148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:00.877148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:00.877148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:00.877148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:00.877148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:00.877148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:00.877148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:00.877148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:00.877148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:00.878030 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:00.878030 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:00.878030 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:00.878030 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:00.878030 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 375/740] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:00.878702 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8df254-8697-4cec-a0c5-1e61fabb1d2e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:00.885726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-fbdd8625-5fd4-49ea-bc29-c7a494d611f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:14:00.885726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:00.885726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:00.885726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:00.885726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:00.885726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:00.885726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:00.885726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:00.885726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:00.885726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:00.885726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:00.885726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:00.885726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:14:00.885726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:14:00.885726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:14:00.885726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:14:00.886722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:14:00.886722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:14:00.886722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:14:00.886722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:14:00.886722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:00.886722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:00.886722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:00.886722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:00.886722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:00.886722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:00.886722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:00.886722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:00.886722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:00.886722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:00.886722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:00.886722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:00.887628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:00.887628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:00.887628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:00.887628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:00.887628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:14:00.887628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:14:00.887628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:14:00.887628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:14:00.887628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:14:00.887628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:14:00.887628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:14:00.887628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:00.887628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 366/741] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 30 16:14:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:00.888792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd8625-5fd4-49ea-bc29-c7a494d611f2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:00.893966 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:00.989375 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8df254-8697-4cec-a0c5-1e61fabb1d2e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:01.002766 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd8625-5fd4-49ea-bc29-c7a494d611f2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:01.032790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb135382-7911-418a-8548-7af108a15513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:01.033977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb135382-7911-418a-8548-7af108a15513 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:01.034640 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb135382-7911-418a-8548-7af108a15513 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:01.035064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb135382-7911-418a-8548-7af108a15513 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:01.036276 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb135382-7911-418a-8548-7af108a15513 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:01.037040 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb135382-7911-418a-8548-7af108a15513 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:01.038430 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb135382-7911-418a-8548-7af108a15513 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:01.041280 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0750c0-d33b-4c10-b0f7-dd29ffa5a534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:01.041955 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0750c0-d33b-4c10-b0f7-dd29ffa5a534 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:01.042240 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0750c0-d33b-4c10-b0f7-dd29ffa5a534 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:01.042520 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0750c0-d33b-4c10-b0f7-dd29ffa5a534 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:01.043246 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0750c0-d33b-4c10-b0f7-dd29ffa5a534 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:01.044674 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0750c0-d33b-4c10-b0f7-dd29ffa5a534 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:01.050440 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 376/742] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:01.051051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb135382-7911-418a-8548-7af108a15513 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:01.092115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:01.126460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 367/743] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 30 16:14:00 2022] POST /identity/v3/services => generated 240 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:14:01.127615 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0750c0-d33b-4c10-b0f7-dd29ffa5a534 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:01.176777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb135382-7911-418a-8548-7af108a15513 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:01.218069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:01.244017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfced873-616a-4a97-b106-0e5a1bd782c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:01.244696 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfced873-616a-4a97-b106-0e5a1bd782c6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:01.244980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfced873-616a-4a97-b106-0e5a1bd782c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:01.245256 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfced873-616a-4a97-b106-0e5a1bd782c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:01.245992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfced873-616a-4a97-b106-0e5a1bd782c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:01.246302 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfced873-616a-4a97-b106-0e5a1bd782c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:01.247775 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfced873-616a-4a97-b106-0e5a1bd782c6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:01.265827 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 377/744] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:01.265827 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfced873-616a-4a97-b106-0e5a1bd782c6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:01.285240 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0750c0-d33b-4c10-b0f7-dd29ffa5a534 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:01.335263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1316aa-e53a-46bd-8648-f8f6929e3171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:01.336383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1316aa-e53a-46bd-8648-f8f6929e3171 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:01.336671 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1316aa-e53a-46bd-8648-f8f6929e3171 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:01.336952 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1316aa-e53a-46bd-8648-f8f6929e3171 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:01.337761 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1316aa-e53a-46bd-8648-f8f6929e3171 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:01.338075 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1316aa-e53a-46bd-8648-f8f6929e3171 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:01.339526 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1316aa-e53a-46bd-8648-f8f6929e3171 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:01.351133 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 368/745] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 30 16:14:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:01.351982 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1316aa-e53a-46bd-8648-f8f6929e3171 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:01.471243 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1316aa-e53a-46bd-8648-f8f6929e3171 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:01.553052 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:01.556639 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:01.565147 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfced873-616a-4a97-b106-0e5a1bd782c6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:01.567312 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e580677-3dff-4e7d-96ab-d73a2daf0480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:01.569689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e580677-3dff-4e7d-96ab-d73a2daf0480 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:01.570089 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e580677-3dff-4e7d-96ab-d73a2daf0480 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:01.570332 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e580677-3dff-4e7d-96ab-d73a2daf0480 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:01.580030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e580677-3dff-4e7d-96ab-d73a2daf0480 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:01.581838 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e580677-3dff-4e7d-96ab-d73a2daf0480 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:01.605823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-4e580677-3dff-4e7d-96ab-d73a2daf0480 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 16:14:01.605823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:01.605823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:01.605823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:01.605823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:01.605823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:01.605823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:01.605823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:01.605823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:01.605823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:01.605823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:01.605823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:01.605823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:14:01.605823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:14:01.605823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:14:01.605823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:14:01.606796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:01.606796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:01.606796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:14:01.606796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:01.606796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:14:01.606796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:01.606796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:01.606796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:01.606796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:01.606796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:01.606796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:01.606796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:01.606796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:01.606796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:01.606796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:01.606796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:01.606796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:01.606796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:01.607779 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:01.607779 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:01.607779 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:14:01.607779 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:14:01.607779 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:14:01.607779 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:14:01.607779 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:14:01.607779 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:14:01.607779 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 16:14:01.607779 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:01.607779 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 369/746] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:14:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:01.608393 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e580677-3dff-4e7d-96ab-d73a2daf0480 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:01.617760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87fa2d-9ae1-4296-9c0f-734e89964963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:01.618891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87fa2d-9ae1-4296-9c0f-734e89964963 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:01.619914 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87fa2d-9ae1-4296-9c0f-734e89964963 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:01.620401 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87fa2d-9ae1-4296-9c0f-734e89964963 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:01.629898 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87fa2d-9ae1-4296-9c0f-734e89964963 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:01.633270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87fa2d-9ae1-4296-9c0f-734e89964963 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:01.652760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-0e87fa2d-9ae1-4296-9c0f-734e89964963 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:01.652760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:01.652760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:01.652760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:01.652760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:01.652760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:01.652760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:01.652760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:01.652760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:01.652760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:01.652760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:01.652760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:01.652760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:14:01.652760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:14:01.652760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:14:01.653693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:14:01.653693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:01.653693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:01.653693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:01.653693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:01.653693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:01.653693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:01.653693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:01.653693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:01.653693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:01.653693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:01.653693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:01.653693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:01.653693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:01.653693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:01.653693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:01.653693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:01.654569 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:14:01.654569 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:14:01.654569 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:14:01.654569 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:14:01.654569 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:14:01.654569 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:14:01.654569 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:01.654569 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:01.657674 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 378/747] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:14:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:01.659290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87fa2d-9ae1-4296-9c0f-734e89964963 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:01.737697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e580677-3dff-4e7d-96ab-d73a2daf0480 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:01.777413 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de119b0-479d-40b0-8a59-6295f7779ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:01.778692 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de119b0-479d-40b0-8a59-6295f7779ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:01.779181 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de119b0-479d-40b0-8a59-6295f7779ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:01.779733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de119b0-479d-40b0-8a59-6295f7779ad9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:01.782386 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87fa2d-9ae1-4296-9c0f-734e89964963 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:01.792693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de119b0-479d-40b0-8a59-6295f7779ad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:01.794888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de119b0-479d-40b0-8a59-6295f7779ad9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:01.814816 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-8de119b0-479d-40b0-8a59-6295f7779ad9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:01.814816 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:01.814816 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:01.814816 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:01.814816 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:01.814816 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:01.814816 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:01.814816 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:01.814816 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:01.814816 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:01.814816 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:01.814816 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:01.814816 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:01.814816 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:01.814816 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:01.815792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:01.815792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:01.815792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:01.815792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:01.815792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:01.815792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:01.815792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:01.815792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:01.815792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:01.815792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:01.815792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:01.815792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:01.815792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:01.815792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:01.815792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:01.815792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:01.815792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:01.816681 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:01.816681 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:01.816681 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:01.816681 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:01.819190 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 370/748] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:01.820297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de119b0-479d-40b0-8a59-6295f7779ad9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:01.824644 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30887d62-d6fc-49f9-af65-11ab9dd8742e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:01.825334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30887d62-d6fc-49f9-af65-11ab9dd8742e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:01.825611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30887d62-d6fc-49f9-af65-11ab9dd8742e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:01.825881 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30887d62-d6fc-49f9-af65-11ab9dd8742e None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:01.826724 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30887d62-d6fc-49f9-af65-11ab9dd8742e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:01.827018 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30887d62-d6fc-49f9-af65-11ab9dd8742e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:01.828788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30887d62-d6fc-49f9-af65-11ab9dd8742e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:01.852977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 379/749] 10.222.0.67 () {56 vars in 1135 bytes} [Thu Jun 30 16:14:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:01.854193 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30887d62-d6fc-49f9-af65-11ab9dd8742e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:01.864172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:01.870226 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:01.948594 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de119b0-479d-40b0-8a59-6295f7779ad9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:01.964710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30887d62-d6fc-49f9-af65-11ab9dd8742e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:01.990324 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7440df-98d4-411d-9ca0-b05d145dcd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:01.991449 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7440df-98d4-411d-9ca0-b05d145dcd7f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:01.992011 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7440df-98d4-411d-9ca0-b05d145dcd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:01.992502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7440df-98d4-411d-9ca0-b05d145dcd7f None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:01.993495 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7440df-98d4-411d-9ca0-b05d145dcd7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:01.994026 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7440df-98d4-411d-9ca0-b05d145dcd7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:01.996019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7440df-98d4-411d-9ca0-b05d145dcd7f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:02.005273 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe99dc-46c7-4d85-8716-12817235ca7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:02.008725 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe99dc-46c7-4d85-8716-12817235ca7d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:02.009012 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe99dc-46c7-4d85-8716-12817235ca7d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:02.009290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fe99dc-46c7-4d85-8716-12817235ca7d None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:02.010034 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe99dc-46c7-4d85-8716-12817235ca7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:02.010340 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe99dc-46c7-4d85-8716-12817235ca7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:02.011744 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fe99dc-46c7-4d85-8716-12817235ca7d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:02.011854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 371/750] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:14:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:02.013331 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7440df-98d4-411d-9ca0-b05d145dcd7f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:02.035274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 380/751] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:02.036177 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe99dc-46c7-4d85-8716-12817235ca7d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:02.114080 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:02.127045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:02.129404 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7440df-98d4-411d-9ca0-b05d145dcd7f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:02.146821 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fe99dc-46c7-4d85-8716-12817235ca7d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:02.167239 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c59f2-996e-4d73-abd6-056d95f2a9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:02.167923 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c59f2-996e-4d73-abd6-056d95f2a9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:02.168208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c59f2-996e-4d73-abd6-056d95f2a9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:02.168483 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c59f2-996e-4d73-abd6-056d95f2a9b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:02.186047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81af0c5-bef0-4618-9698-215d91d14241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:02.186937 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81af0c5-bef0-4618-9698-215d91d14241 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:02.187213 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81af0c5-bef0-4618-9698-215d91d14241 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:02.187531 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81af0c5-bef0-4618-9698-215d91d14241 None admin] PATH_INFO: `/v3/projects/19ef92ebcce94b90bf39ad3a58772a1d/users/71cca3708386459197abf6c67505bc5f/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:02.189461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81af0c5-bef0-4618-9698-215d91d14241 None admin] RBAC: Authorizing `identity:create_grant(project_id=19ef92ebcce94b90bf39ad3a58772a1d, user_id=71cca3708386459197abf6c67505bc5f, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:02.190912 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81af0c5-bef0-4618-9698-215d91d14241 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:02.241929 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c59f2-996e-4d73-abd6-056d95f2a9b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:02.243389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c59f2-996e-4d73-abd6-056d95f2a9b1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:02.279997 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 381/752] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 30 16:14:02 2022] PUT /identity/v3/projects/19ef92ebcce94b90bf39ad3a58772a1d/users/71cca3708386459197abf6c67505bc5f/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:14:02.281192 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81af0c5-bef0-4618-9698-215d91d14241 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:02.370196 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:02.381878 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 372/753] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:14:02 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 16:14:02.384187 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acaaebe-f7f4-4378-82a5-bdd5d10ce433 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:02.384478 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acaaebe-f7f4-4378-82a5-bdd5d10ce433 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:02.384758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acaaebe-f7f4-4378-82a5-bdd5d10ce433 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:02.387271 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 373/754] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:02.388408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acaaebe-f7f4-4378-82a5-bdd5d10ce433 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:02.440176 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81af0c5-bef0-4618-9698-215d91d14241 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:02.481081 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:02.486221 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be6071-eaeb-44ea-8f92-dd2e7dbf6997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:02.487182 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be6071-eaeb-44ea-8f92-dd2e7dbf6997 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:02.487479 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be6071-eaeb-44ea-8f92-dd2e7dbf6997 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:02.487820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be6071-eaeb-44ea-8f92-dd2e7dbf6997 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:02.496973 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be6071-eaeb-44ea-8f92-dd2e7dbf6997 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:02.498826 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be6071-eaeb-44ea-8f92-dd2e7dbf6997 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:02.514457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f4be6071-eaeb-44ea-8f92-dd2e7dbf6997 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 16:14:02.514457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:02.514457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:02.514457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:02.514457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:02.514457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:02.514457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:02.514457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:02.514457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:02.514457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:02.514457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:02.514457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:02.514457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:14:02.514457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:14:02.514457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:14:02.514457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:14:02.515443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:02.515443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:02.515443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:14:02.515443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:02.515443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:14:02.515443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:02.515443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:02.515443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:02.515443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:02.515443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:02.515443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:02.515443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:02.515443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:02.515443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:02.515443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:02.515443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:02.515443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:02.515443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:02.516539 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:02.516539 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:02.516539 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:14:02.516539 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:14:02.516539 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:14:02.516539 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:14:02.516539 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:14:02.516539 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:14:02.516539 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 16:14:02.516539 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:02.517063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 382/755] 10.222.0.67 () {56 vars in 1117 bytes} [Thu Jun 30 16:14:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:02.520500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9b504-7d26-40c8-ad64-0a7d49018d29 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:02.520500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9b504-7d26-40c8-ad64-0a7d49018d29 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:02.520500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9b504-7d26-40c8-ad64-0a7d49018d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:02.538993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8dc9b504-7d26-40c8-ad64-0a7d49018d29 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:02.540650 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8dc9b504-7d26-40c8-ad64-0a7d49018d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:02.556954 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acaaebe-f7f4-4378-82a5-bdd5d10ce433 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:02.596082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20409a-c695-4b30-8784-25c68b4d3cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:02.596986 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20409a-c695-4b30-8784-25c68b4d3cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:02.597333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20409a-c695-4b30-8784-25c68b4d3cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:02.597676 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20409a-c695-4b30-8784-25c68b4d3cf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:02.608892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20409a-c695-4b30-8784-25c68b4d3cf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:02.610486 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20409a-c695-4b30-8784-25c68b4d3cf4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:02.623595 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 383/756] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:02.624770 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9b504-7d26-40c8-ad64-0a7d49018d29 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:02.629208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-6b20409a-c695-4b30-8784-25c68b4d3cf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:02.629208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:02.629208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:02.629208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:02.629208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:02.629208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:02.629208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:02.629208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:02.629208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:02.629208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:02.629208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:02.629208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:02.629208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:02.629208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:02.629208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:02.631008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:02.631008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:02.631008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:02.631008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:02.631008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:02.631008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:02.631008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:02.631008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:02.631008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:02.631008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:02.631008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:02.631008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:02.631008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:02.631008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:02.631008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:02.631008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:02.631008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:02.632285 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:02.632285 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:02.632285 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:02.632285 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:02.638358 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 374/757] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:02.639730 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20409a-c695-4b30-8784-25c68b4d3cf4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:02.711749 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:02.768910 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9b504-7d26-40c8-ad64-0a7d49018d29 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:02.778565 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20409a-c695-4b30-8784-25c68b4d3cf4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:02.799833 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:02.816626 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76287f6e-7e3b-4050-954d-2b296216ac7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:02.818694 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76287f6e-7e3b-4050-954d-2b296216ac7a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:02.819418 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76287f6e-7e3b-4050-954d-2b296216ac7a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:02.819959 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76287f6e-7e3b-4050-954d-2b296216ac7a None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:02.821344 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76287f6e-7e3b-4050-954d-2b296216ac7a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:02.821924 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76287f6e-7e3b-4050-954d-2b296216ac7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:02.823664 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8bc40c-9dac-4cda-8fab-b6c89527a673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:02.824002 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76287f6e-7e3b-4050-954d-2b296216ac7a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:02.824563 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8bc40c-9dac-4cda-8fab-b6c89527a673 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:02.824946 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8bc40c-9dac-4cda-8fab-b6c89527a673 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:02.825325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8bc40c-9dac-4cda-8fab-b6c89527a673 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:02.826173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8bc40c-9dac-4cda-8fab-b6c89527a673 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:02.826581 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8bc40c-9dac-4cda-8fab-b6c89527a673 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:02.828255 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8bc40c-9dac-4cda-8fab-b6c89527a673 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:02.844921 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 384/759] 10.222.0.67 () {56 vars in 1149 bytes} [Thu Jun 30 16:14:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:02.846139 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76287f6e-7e3b-4050-954d-2b296216ac7a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:02.847722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 375/759] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:02.848668 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8bc40c-9dac-4cda-8fab-b6c89527a673 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:02.957305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8bc40c-9dac-4cda-8fab-b6c89527a673 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:02.960661 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76287f6e-7e3b-4050-954d-2b296216ac7a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:02.967203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:03.006247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6c19b-430b-4edb-854c-99ad2c0f3d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:03.007393 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6c19b-430b-4edb-854c-99ad2c0f3d5c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:03.007844 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6c19b-430b-4edb-854c-99ad2c0f3d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:03.008262 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6c19b-430b-4edb-854c-99ad2c0f3d5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:03.010595 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21084b-1b06-4b1c-b00f-0e81d03db217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:03.011812 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21084b-1b06-4b1c-b00f-0e81d03db217 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:03.012573 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21084b-1b06-4b1c-b00f-0e81d03db217 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:03.012958 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21084b-1b06-4b1c-b00f-0e81d03db217 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:03.013736 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21084b-1b06-4b1c-b00f-0e81d03db217 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:03.015357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21084b-1b06-4b1c-b00f-0e81d03db217 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:03.017558 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b6c19b-430b-4edb-854c-99ad2c0f3d5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:03.020308 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b6c19b-430b-4edb-854c-99ad2c0f3d5c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:03.031167 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-ae21084b-1b06-4b1c-b00f-0e81d03db217 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 16:14:03.031167 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:03.031167 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:14:03.031167 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:14:03.031167 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:14:03.031167 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:14:03.031167 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:14:03.031167 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:03.031167 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 16:14:03.031167 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:03.031167 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:14:03.031167 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:03.031167 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:03.031167 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:03.031167 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:03.031167 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:03.032207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:03.032207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:03.032207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:03.032207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:03.032207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:03.032207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:03.032207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:03.032207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:14:03.032207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:14:03.032207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:14:03.032207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:14:03.032207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:03.032207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:03.032207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:03.032207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:03.032207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:03.033039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:03.033039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:03.033039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:03.033039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:03.033039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:03.033039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:03.033039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:03.033039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:03.033039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:03.033039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:03.033039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:03.033039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:14:03.033039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:03.033039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 16:14:03.033039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:03.038029 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 385/760] 10.222.0.67 () {56 vars in 1129 bytes} [Thu Jun 30 16:14:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:03.038999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21084b-1b06-4b1c-b00f-0e81d03db217 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:03.040920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-57b6c19b-430b-4edb-854c-99ad2c0f3d5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:03.040920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:03.040920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:03.040920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:03.040920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:03.040920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:03.040920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:03.040920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:03.040920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:03.040920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:03.040920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:03.040920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:03.040920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:14:03.040920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:14:03.040920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:14:03.041801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:14:03.041801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:03.041801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:03.041801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:03.041801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:03.041801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:03.041801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:03.041801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:03.041801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:03.041801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:03.041801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:03.041801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:03.041801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:03.041801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:03.041801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:03.041801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:03.041801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:03.042662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:14:03.042662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:14:03.042662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:14:03.042662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:14:03.042662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:14:03.042662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:14:03.042662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:03.042662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:03.046165 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 376/761] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:14:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:03.047222 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6c19b-430b-4edb-854c-99ad2c0f3d5c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:03.096196 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:03.155260 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21084b-1b06-4b1c-b00f-0e81d03db217 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:03.158575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6c19b-430b-4edb-854c-99ad2c0f3d5c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:03.191833 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48edfb4a-6908-426b-840f-9e5f669042ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:03.193559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edfb4a-6908-426b-840f-9e5f669042ca None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:03.194138 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edfb4a-6908-426b-840f-9e5f669042ca None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:03.194628 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48edfb4a-6908-426b-840f-9e5f669042ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:03.196204 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e2146-3e15-4c9b-a7fc-8bd0c872c95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:03.196947 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e2146-3e15-4c9b-a7fc-8bd0c872c95b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:03.197303 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e2146-3e15-4c9b-a7fc-8bd0c872c95b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:03.197650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e2146-3e15-4c9b-a7fc-8bd0c872c95b None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:03.198497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e2146-3e15-4c9b-a7fc-8bd0c872c95b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:03.198799 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e2146-3e15-4c9b-a7fc-8bd0c872c95b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:03.200644 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e2146-3e15-4c9b-a7fc-8bd0c872c95b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:03.211081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 377/762] 10.222.0.67 () {56 vars in 1124 bytes} [Thu Jun 30 16:14:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:03.212057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e2146-3e15-4c9b-a7fc-8bd0c872c95b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:03.245453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:03.272093 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48edfb4a-6908-426b-840f-9e5f669042ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:03.274281 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48edfb4a-6908-426b-840f-9e5f669042ca None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:03.327764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:03.390391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 386/763] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:14:03 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 16:14:03.391642 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48edfb4a-6908-426b-840f-9e5f669042ca None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:03.439906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e2146-3e15-4c9b-a7fc-8bd0c872c95b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:03.476949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91c26b-8c4b-488a-a536-c687a4ed2a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:03.477545 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91c26b-8c4b-488a-a536-c687a4ed2a4c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:03.477827 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91c26b-8c4b-488a-a536-c687a4ed2a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:03.478103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91c26b-8c4b-488a-a536-c687a4ed2a4c None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:03.478932 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91c26b-8c4b-488a-a536-c687a4ed2a4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:03.479243 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91c26b-8c4b-488a-a536-c687a4ed2a4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:03.480732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91c26b-8c4b-488a-a536-c687a4ed2a4c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:03.493055 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 378/764] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:14:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:03.494370 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe35f0-c695-4bc9-9c44-cbac83cd64a8 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:03.494654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe35f0-c695-4bc9-9c44-cbac83cd64a8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:03.494929 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe35f0-c695-4bc9-9c44-cbac83cd64a8 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:03.499503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 379/765] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:03.499503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe35f0-c695-4bc9-9c44-cbac83cd64a8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:03.511788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48edfb4a-6908-426b-840f-9e5f669042ca None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:03.560209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677c48c-69fd-4942-8de4-e0c6f2f5b78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:03.560879 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677c48c-69fd-4942-8de4-e0c6f2f5b78f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:03.561161 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677c48c-69fd-4942-8de4-e0c6f2f5b78f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:03.561560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677c48c-69fd-4942-8de4-e0c6f2f5b78f None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:03.562284 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677c48c-69fd-4942-8de4-e0c6f2f5b78f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:03.562597 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677c48c-69fd-4942-8de4-e0c6f2f5b78f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:03.564026 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677c48c-69fd-4942-8de4-e0c6f2f5b78f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:03.570111 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 387/766] 10.222.0.67 () {56 vars in 1124 bytes} [Thu Jun 30 16:14:03 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:03.575569 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677c48c-69fd-4942-8de4-e0c6f2f5b78f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:03.586045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:03.599392 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:03.610460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe35f0-c695-4bc9-9c44-cbac83cd64a8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:03.647385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300397d-33db-4f9c-b04d-14b277cc528f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:03.648464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300397d-33db-4f9c-b04d-14b277cc528f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:03.648752 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300397d-33db-4f9c-b04d-14b277cc528f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:03.649027 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300397d-33db-4f9c-b04d-14b277cc528f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:03.659538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300397d-33db-4f9c-b04d-14b277cc528f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:03.661384 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300397d-33db-4f9c-b04d-14b277cc528f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:03.682325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-2300397d-33db-4f9c-b04d-14b277cc528f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:03.682325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:03.682325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:03.682325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:03.682325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:03.682325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:03.682325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:03.682325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:03.682325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:03.682325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:03.682325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:03.682325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:03.682325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:03.682325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:03.682325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:03.683556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:03.683556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:03.683556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:03.683556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:03.683556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:03.683556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:03.683556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:03.683556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:03.683556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:03.683556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:03.683556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:03.683556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:03.683556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:03.683556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:03.683556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:03.683556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:03.683556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:03.684780 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:03.684780 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:03.684780 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:03.684780 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:03.685062 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677c48c-69fd-4942-8de4-e0c6f2f5b78f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:03.686398 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 380/767] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:03.687761 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11c0ab-7b98-44ff-bd56-1d79c45fecb0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:03.688191 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11c0ab-7b98-44ff-bd56-1d79c45fecb0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:03.688595 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11c0ab-7b98-44ff-bd56-1d79c45fecb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:03.705743 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5e11c0ab-7b98-44ff-bd56-1d79c45fecb0 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:03.707098 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5e11c0ab-7b98-44ff-bd56-1d79c45fecb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:03.727028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f41524-6741-40ec-a27e-7f7b099e968a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:03.727911 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f41524-6741-40ec-a27e-7f7b099e968a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:03.728368 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f41524-6741-40ec-a27e-7f7b099e968a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:03.728912 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f41524-6741-40ec-a27e-7f7b099e968a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:03.729744 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f41524-6741-40ec-a27e-7f7b099e968a None admin] RBAC: Adding query filter params (role.id=8acc34a163874a60b03728b814eaf075, scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d, user.id=988b917eea4b48ffae32e23b8555a5c6) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:03.730102 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f41524-6741-40ec-a27e-7f7b099e968a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:03.731754 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f41524-6741-40ec-a27e-7f7b099e968a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:03.747502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 388/768] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:14:03 2022] GET /identity/v3/role_assignments?user.id=988b917eea4b48ffae32e23b8555a5c6&scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d&role.id=8acc34a163874a60b03728b814eaf075 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:03.747502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f41524-6741-40ec-a27e-7f7b099e968a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:03.799669 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 381/769] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:03.800650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11c0ab-7b98-44ff-bd56-1d79c45fecb0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:03.847683 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:03.882315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f41524-6741-40ec-a27e-7f7b099e968a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:03.926488 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:03.928620 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d2295-8fac-4de1-8739-deec9d66202f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:03.929477 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d2295-8fac-4de1-8739-deec9d66202f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:03.929477 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d2295-8fac-4de1-8739-deec9d66202f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:03.929884 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d2295-8fac-4de1-8739-deec9d66202f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:03.930682 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d2295-8fac-4de1-8739-deec9d66202f None admin] RBAC: Adding query filter params (interface=public, service_id=a22014b021e44bf1a219051382666053, region_id=RegionOne) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:03.931006 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d2295-8fac-4de1-8739-deec9d66202f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:03.932424 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6d2295-8fac-4de1-8739-deec9d66202f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:03.942485 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 389/770] 10.222.0.67 () {56 vars in 1251 bytes} [Thu Jun 30 16:14:03 2022] GET /identity/v3/endpoints?service_id=a22014b021e44bf1a219051382666053&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:03.943575 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d2295-8fac-4de1-8739-deec9d66202f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:03.967104 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11c0ab-7b98-44ff-bd56-1d79c45fecb0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:04.011009 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d519cc1-99af-4039-803d-219249485710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:04.011955 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d519cc1-99af-4039-803d-219249485710 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:04.012238 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d519cc1-99af-4039-803d-219249485710 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:04.012517 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d519cc1-99af-4039-803d-219249485710 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:04.013575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d519cc1-99af-4039-803d-219249485710 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:04.013892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d519cc1-99af-4039-803d-219249485710 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:04.015353 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d519cc1-99af-4039-803d-219249485710 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:04.035169 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 382/771] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:04.035169 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d519cc1-99af-4039-803d-219249485710 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:04.080538 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6d2295-8fac-4de1-8739-deec9d66202f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:04.121819 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56d454-134b-4002-a4cc-22163dc45ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:04.122979 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56d454-134b-4002-a4cc-22163dc45ded None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:04.123503 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56d454-134b-4002-a4cc-22163dc45ded None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:04.124033 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56d454-134b-4002-a4cc-22163dc45ded None admin] PATH_INFO: `/v3/services/network` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:04.124897 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de56d454-134b-4002-a4cc-22163dc45ded None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:04.126495 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de56d454-134b-4002-a4cc-22163dc45ded None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:04.129857 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:04.141527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-de56d454-134b-4002-a4cc-22163dc45ded None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 16:14:04.141527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:04.141527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:14:04.141527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:14:04.141527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:14:04.141527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:14:04.141527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:14:04.141527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:04.141527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 16:14:04.141527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:04.141527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:14:04.141527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:04.141527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:04.141527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:04.141527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:04.141527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:04.141527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:04.161095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:04.161095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:04.161095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:04.161095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:04.161095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:04.161095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:04.161095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:14:04.161095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:14:04.161095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:14:04.161095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:14:04.161095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:04.161095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:04.161095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:04.161095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:04.161095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:04.161095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:04.162513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:04.162513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:04.162513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:04.162513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:04.162513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:04.162513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:04.162513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:04.162513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:04.162513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:04.162513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:04.162513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:14:04.162513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:04.162513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 16:14:04.162513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:04.162513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 390/772] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:14:03 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:04.163320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56d454-134b-4002-a4cc-22163dc45ded None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:04.163398 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d519cc1-99af-4039-803d-219249485710 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:04.190800 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09251945-0baf-4994-be26-9dc4115197f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:04.192429 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09251945-0baf-4994-be26-9dc4115197f6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:04.192429 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09251945-0baf-4994-be26-9dc4115197f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:04.192429 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09251945-0baf-4994-be26-9dc4115197f6 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:04.193223 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09251945-0baf-4994-be26-9dc4115197f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:04.194688 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09251945-0baf-4994-be26-9dc4115197f6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:04.196002 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:04.206907 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 383/773] 10.222.0.67 () {56 vars in 1087 bytes} [Thu Jun 30 16:14:04 2022] GET /identity/v3/services => generated 772 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:04.206907 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09251945-0baf-4994-be26-9dc4115197f6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:04.294604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56d454-134b-4002-a4cc-22163dc45ded None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:04.324769 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09251945-0baf-4994-be26-9dc4115197f6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:04.335077 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d3637-b343-4309-ad14-4c588e9a551f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:04.336268 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d3637-b343-4309-ad14-4c588e9a551f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:04.336765 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d3637-b343-4309-ad14-4c588e9a551f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:04.337252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d3637-b343-4309-ad14-4c588e9a551f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:04.345839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2d3637-b343-4309-ad14-4c588e9a551f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:04.347908 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2d3637-b343-4309-ad14-4c588e9a551f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:04.377634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-cf2d3637-b343-4309-ad14-4c588e9a551f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:04.377634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:04.377634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:04.377634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:04.377634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:04.377634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:04.377634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:04.377634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:04.377634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:04.377634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:04.377634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:04.377634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:04.377634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:14:04.377634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:14:04.377634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:14:04.378583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:14:04.378583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:04.378583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:04.378583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:04.378583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:04.378583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:04.378583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:04.378583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:04.378583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:04.378583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:04.378583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:04.378583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:04.378583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:04.378583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:04.378583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:04.378583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:04.378583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:04.379500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:14:04.379500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:14:04.379500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:14:04.379500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:14:04.379500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:14:04.379500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:14:04.379500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:04.379500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:04.382089 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 391/774] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:14:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:04.388294 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d3637-b343-4309-ad14-4c588e9a551f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:04.403051 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6b710-e011-4101-88bc-16164b60f691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:04.403804 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6b710-e011-4101-88bc-16164b60f691 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:04.404081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6b710-e011-4101-88bc-16164b60f691 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:04.404380 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a6b710-e011-4101-88bc-16164b60f691 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:04.405343 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6b710-e011-4101-88bc-16164b60f691 None admin] RBAC: Adding query filter params (name=network) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:04.405525 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6b710-e011-4101-88bc-16164b60f691 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:04.406889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a6b710-e011-4101-88bc-16164b60f691 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:04.419189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 384/775] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 30 16:14:04 2022] GET /identity/v3/services?name=network => generated 124 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:04.429265 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6b710-e011-4101-88bc-16164b60f691 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:04.467932 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:04.520001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:04.520197 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d3637-b343-4309-ad14-4c588e9a551f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:04.562345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9ca3c-9ba3-4c01-afd0-a19112357a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:04.563013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9ca3c-9ba3-4c01-afd0-a19112357a72 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:04.563302 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9ca3c-9ba3-4c01-afd0-a19112357a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:04.563636 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9ca3c-9ba3-4c01-afd0-a19112357a72 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:04.564678 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9ca3c-9ba3-4c01-afd0-a19112357a72 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:04.565008 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9ca3c-9ba3-4c01-afd0-a19112357a72 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:04.566459 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9ca3c-9ba3-4c01-afd0-a19112357a72 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:04.573800 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a6b710-e011-4101-88bc-16164b60f691 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:04.580513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 392/776] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:14:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:04.587593 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0cf3a-9663-4889-bf36-47f8ddbc4816 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:04.587972 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0cf3a-9663-4889-bf36-47f8ddbc4816 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:04.588300 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0cf3a-9663-4889-bf36-47f8ddbc4816 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:04.590111 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 393/777] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:04.590867 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f0cf3a-9663-4889-bf36-47f8ddbc4816 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:04.618680 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7a954-065e-48e4-ac23-c03697b52908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:04.619367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7a954-065e-48e4-ac23-c03697b52908 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:04.619667 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7a954-065e-48e4-ac23-c03697b52908 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:04.619957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7a954-065e-48e4-ac23-c03697b52908 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:04.620652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7a954-065e-48e4-ac23-c03697b52908 None admin] RBAC: Adding query filter params (type=network) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:04.620958 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7a954-065e-48e4-ac23-c03697b52908 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:04.622334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7a954-065e-48e4-ac23-c03697b52908 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:04.627805 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 385/778] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 30 16:14:04 2022] GET /identity/v3/services?type=network => generated 350 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:04.629094 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175501da-48a6-467f-99f2-0890019ec398 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:04.629454 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175501da-48a6-467f-99f2-0890019ec398 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:04.629657 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175501da-48a6-467f-99f2-0890019ec398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:04.647805 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-175501da-48a6-467f-99f2-0890019ec398 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:04.649727 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-175501da-48a6-467f-99f2-0890019ec398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:04.703276 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f0cf3a-9663-4889-bf36-47f8ddbc4816 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:04.725451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:04.741958 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9383d-4817-4d46-8d1d-1bf1047ec8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:04.743137 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9383d-4817-4d46-8d1d-1bf1047ec8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:04.743728 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9383d-4817-4d46-8d1d-1bf1047ec8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:04.744276 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c9383d-4817-4d46-8d1d-1bf1047ec8d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:04.745381 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c9383d-4817-4d46-8d1d-1bf1047ec8d9 None admin] RBAC: Adding query filter params (role.id=57bc5df632b3407d97116ae34d92a5cb, scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d, user.id=d49604bffa3d4e89a631eefa6d4e4f5d) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:04.745953 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c9383d-4817-4d46-8d1d-1bf1047ec8d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:04.747304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 386/779] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:04.747911 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c9383d-4817-4d46-8d1d-1bf1047ec8d9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:04.748229 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175501da-48a6-467f-99f2-0890019ec398 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:04.762937 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 394/780] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:14:04 2022] GET /identity/v3/role_assignments?user.id=d49604bffa3d4e89a631eefa6d4e4f5d&scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d&role.id=57bc5df632b3407d97116ae34d92a5cb => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:04.776798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9383d-4817-4d46-8d1d-1bf1047ec8d9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:04.861645 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:04.922841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175501da-48a6-467f-99f2-0890019ec398 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:04.962090 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa541f9-ede2-414c-ba3d-fa0b428bca9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:04.963108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa541f9-ede2-414c-ba3d-fa0b428bca9b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:04.963629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa541f9-ede2-414c-ba3d-fa0b428bca9b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:04.964096 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa541f9-ede2-414c-ba3d-fa0b428bca9b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:04.965167 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa541f9-ede2-414c-ba3d-fa0b428bca9b None admin] RBAC: Adding query filter params (interface=public, service_id=51e99a86afbe405fb7bc358a1c42cd06, region_id=RegionOne) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:04.965799 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa541f9-ede2-414c-ba3d-fa0b428bca9b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:04.967357 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa541f9-ede2-414c-ba3d-fa0b428bca9b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:04.976198 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 387/781] 10.222.0.67 () {56 vars in 1251 bytes} [Thu Jun 30 16:14:04 2022] GET /identity/v3/endpoints?service_id=51e99a86afbe405fb7bc358a1c42cd06&interface=public®ion_id=RegionOne => generated 194 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:04.984465 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa541f9-ede2-414c-ba3d-fa0b428bca9b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:05.033603 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c9383d-4817-4d46-8d1d-1bf1047ec8d9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:05.106412 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d241842-4661-4523-8ef7-3b0c811895c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:05.107267 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d241842-4661-4523-8ef7-3b0c811895c6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:05.107570 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d241842-4661-4523-8ef7-3b0c811895c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:05.107842 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d241842-4661-4523-8ef7-3b0c811895c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:05.136055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d241842-4661-4523-8ef7-3b0c811895c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:14:05.143732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-4d241842-4661-4523-8ef7-3b0c811895c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:14:05.143732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:05.143732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:05.143732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:05.143732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:05.143732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:05.143732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:05.143732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:05.143732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:05.143732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:05.143732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:05.143732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:05.143732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:14:05.143732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:14:05.143732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:14:05.143732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:14:05.144673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:14:05.144673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:14:05.144673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:14:05.144673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:14:05.144673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:05.144673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:05.144673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:05.144673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:05.144673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:05.144673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:05.144673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:05.144673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:05.144673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:05.144673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:05.144673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:05.144673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:05.145604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:05.145604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:05.145604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:05.145604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:05.145604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:14:05.145604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:14:05.145604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:14:05.145604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:14:05.145604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:14:05.145604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:14:05.145604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:14:05.145604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:05.148443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 395/782] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:14:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:05.154412 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d241842-4661-4523-8ef7-3b0c811895c6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:05.188342 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa541f9-ede2-414c-ba3d-fa0b428bca9b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:05.223415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:05.243847 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:05.253196 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41cd05-1f2f-4735-837a-a0429124001c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:05.253863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41cd05-1f2f-4735-837a-a0429124001c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:05.254145 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41cd05-1f2f-4735-837a-a0429124001c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:05.254419 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41cd05-1f2f-4735-837a-a0429124001c None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:05.255800 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41cd05-1f2f-4735-837a-a0429124001c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:05.257216 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41cd05-1f2f-4735-837a-a0429124001c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:05.265577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 388/783] 10.222.0.67 () {56 vars in 1087 bytes} [Thu Jun 30 16:14:04 2022] GET /identity/v3/services => generated 772 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:05.348441 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d241842-4661-4523-8ef7-3b0c811895c6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:05.387616 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad899b-737f-4a97-93fa-aacca8a65818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:05.388216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad899b-737f-4a97-93fa-aacca8a65818 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:05.388499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad899b-737f-4a97-93fa-aacca8a65818 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:05.388771 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad899b-737f-4a97-93fa-aacca8a65818 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:05.389525 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad899b-737f-4a97-93fa-aacca8a65818 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:05.389871 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad899b-737f-4a97-93fa-aacca8a65818 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:05.391268 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad899b-737f-4a97-93fa-aacca8a65818 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:05.401356 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 396/784] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:14:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:05.408739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41cd05-1f2f-4735-837a-a0429124001c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:05.523026 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41cd05-1f2f-4735-837a-a0429124001c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:05.565159 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997dc5b-37d2-4b7f-a3c2-9ab7bda98524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:05.566553 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997dc5b-37d2-4b7f-a3c2-9ab7bda98524 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:05.566986 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997dc5b-37d2-4b7f-a3c2-9ab7bda98524 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:05.567440 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e997dc5b-37d2-4b7f-a3c2-9ab7bda98524 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:05.578990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e997dc5b-37d2-4b7f-a3c2-9ab7bda98524 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:05.580678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e997dc5b-37d2-4b7f-a3c2-9ab7bda98524 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:05.605383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e997dc5b-37d2-4b7f-a3c2-9ab7bda98524 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:05.605383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:05.605383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:05.605383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:05.605383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:05.605383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:05.605383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:05.605383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:05.605383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:05.605383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:05.605383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:05.605383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:05.605383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:05.605383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:05.605383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:05.606502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:05.606502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:05.606502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:05.606502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:05.606502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:05.606502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:05.606502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:05.606502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:05.606502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:05.606502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:05.606502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:05.606502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:05.606502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:05.606502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:05.606502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:05.606502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:05.606502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:05.607574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:05.607574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:05.607574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:05.607574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:05.609583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 389/785] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:05.619968 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad899b-737f-4a97-93fa-aacca8a65818 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:05.768951 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad899b-737f-4a97-93fa-aacca8a65818 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:05.777722 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:05.819689 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70dedf6-055e-46c6-bf81-3e2c42810af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:05.820359 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70dedf6-055e-46c6-bf81-3e2c42810af4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:05.820639 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70dedf6-055e-46c6-bf81-3e2c42810af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:05.820910 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70dedf6-055e-46c6-bf81-3e2c42810af4 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:05.821637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70dedf6-055e-46c6-bf81-3e2c42810af4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:05.822188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70dedf6-055e-46c6-bf81-3e2c42810af4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:05.823771 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70dedf6-055e-46c6-bf81-3e2c42810af4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:05.835867 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 397/786] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:05.841171 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997dc5b-37d2-4b7f-a3c2-9ab7bda98524 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:05.915092 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:05.987912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e997dc5b-37d2-4b7f-a3c2-9ab7bda98524 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:06.031007 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a5936-c008-43c3-b267-80b2474bd40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:06.031758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a5936-c008-43c3-b267-80b2474bd40e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:06.032043 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a5936-c008-43c3-b267-80b2474bd40e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:06.032316 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a5936-c008-43c3-b267-80b2474bd40e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:06.040409 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a5936-c008-43c3-b267-80b2474bd40e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:06.041857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a5936-c008-43c3-b267-80b2474bd40e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:06.065179 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-6e8a5936-c008-43c3-b267-80b2474bd40e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 16:14:06.065179 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:06.065179 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:06.065179 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:06.065179 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:06.065179 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:06.065179 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:06.065179 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:06.065179 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:06.065179 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:06.065179 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:06.065179 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:06.065179 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:14:06.065179 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:14:06.065179 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:14:06.065179 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:14:06.066149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:06.066149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:06.066149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:14:06.066149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:06.066149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:14:06.066149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:06.066149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:06.066149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:06.066149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:06.066149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:06.066149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:06.066149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:06.066149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:06.066149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:06.066149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:06.066149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:06.066149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:06.066149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:06.067159 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:06.067159 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:06.067159 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:14:06.067159 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:14:06.067159 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:14:06.067159 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:14:06.067159 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:14:06.067159 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:14:06.067159 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 16:14:06.067159 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:06.067159 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 390/787] 10.222.0.67 () {56 vars in 1093 bytes} [Thu Jun 30 16:14:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:06.075204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70dedf6-055e-46c6-bf81-3e2c42810af4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:06.195164 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70dedf6-055e-46c6-bf81-3e2c42810af4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:06.221606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2aa2b-3e37-47cd-8d96-b581793ee808 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:06.222119 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2aa2b-3e37-47cd-8d96-b581793ee808 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:06.222193 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2aa2b-3e37-47cd-8d96-b581793ee808 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:06.224867 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 391/788] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:06.231608 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647dafd-4c14-447b-9253-bea298a57a11 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:06.231608 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647dafd-4c14-447b-9253-bea298a57a11 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:06.231608 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647dafd-4c14-447b-9253-bea298a57a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:06.237191 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a994364-c23c-4008-bda6-5ca2adfe2111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:06.237884 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a994364-c23c-4008-bda6-5ca2adfe2111 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:06.238192 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a994364-c23c-4008-bda6-5ca2adfe2111 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:06.238470 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a994364-c23c-4008-bda6-5ca2adfe2111 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:06.243517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a994364-c23c-4008-bda6-5ca2adfe2111 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:06.243517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a994364-c23c-4008-bda6-5ca2adfe2111 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:06.243517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a994364-c23c-4008-bda6-5ca2adfe2111 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:06.262049 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1647dafd-4c14-447b-9253-bea298a57a11 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:06.263293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1647dafd-4c14-447b-9253-bea298a57a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:06.280161 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 398/789] 10.222.0.67 () {56 vars in 1133 bytes} [Thu Jun 30 16:14:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:06.288587 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a994364-c23c-4008-bda6-5ca2adfe2111 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:06.352566 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:06.355946 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 392/790] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 16:14:06.375268 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647dafd-4c14-447b-9253-bea298a57a11 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:06.407851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:06.488591 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647dafd-4c14-447b-9253-bea298a57a11 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:06.529712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9adae5-1a5b-420b-a986-bdf2e4dfa50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:06.530785 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9adae5-1a5b-420b-a986-bdf2e4dfa50a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:06.531218 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9adae5-1a5b-420b-a986-bdf2e4dfa50a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:06.531670 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9adae5-1a5b-420b-a986-bdf2e4dfa50a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:06.537680 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a994364-c23c-4008-bda6-5ca2adfe2111 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:06.545185 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9adae5-1a5b-420b-a986-bdf2e4dfa50a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:14:06.554039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-5a9adae5-1a5b-420b-a986-bdf2e4dfa50a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:14:06.554039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:06.554039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:06.554039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:06.554039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:06.554039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:06.554039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:06.554039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:06.554039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:06.554039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:06.554039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:06.554039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:06.554039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:14:06.554039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:14:06.554039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:14:06.554039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:14:06.555173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:14:06.555173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:14:06.555173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:14:06.555173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:14:06.555173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:06.555173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:06.555173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:06.555173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:06.555173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:06.555173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:06.555173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:06.555173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:06.555173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:06.555173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:06.555173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:06.555173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:06.556238 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:06.556238 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:06.556238 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:06.556238 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:06.556238 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:14:06.556238 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:14:06.556238 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:14:06.556238 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:14:06.556238 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:14:06.556238 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:14:06.556238 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:14:06.556238 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:06.564436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 393/791] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:14:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:14:06.566309 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9adae5-1a5b-420b-a986-bdf2e4dfa50a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:06.633106 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a75b4-86ba-4d72-8f75-c9756164e960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:06.633783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a75b4-86ba-4d72-8f75-c9756164e960 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:06.634048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a75b4-86ba-4d72-8f75-c9756164e960 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:06.634324 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a75b4-86ba-4d72-8f75-c9756164e960 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:06.663583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:06.680649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9adae5-1a5b-420b-a986-bdf2e4dfa50a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:06.721155 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce11898a-7211-436d-8058-875d2f88998b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:06.722302 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11898a-7211-436d-8058-875d2f88998b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:06.722789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11898a-7211-436d-8058-875d2f88998b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:06.723323 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11898a-7211-436d-8058-875d2f88998b None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:06.725200 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce11898a-7211-436d-8058-875d2f88998b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:06.725751 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce11898a-7211-436d-8058-875d2f88998b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:06.728671 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce11898a-7211-436d-8058-875d2f88998b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:06.744094 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 394/792] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:14:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:06.747886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce11898a-7211-436d-8058-875d2f88998b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:06.775272 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a75b4-86ba-4d72-8f75-c9756164e960 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:06.776878 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a75b4-86ba-4d72-8f75-c9756164e960 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:06.802964 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:06.872760 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce11898a-7211-436d-8058-875d2f88998b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:06.912068 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 399/793] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:14:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 16:14:06.912068 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475216ba-1e7b-4b13-9d5b-5b353e337b65 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:06.912068 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475216ba-1e7b-4b13-9d5b-5b353e337b65 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:06.912068 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475216ba-1e7b-4b13-9d5b-5b353e337b65 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:06.915285 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 400/794] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:06.916682 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d072217-ded2-4bcb-b82c-6520afc448a5 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:06.917044 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d072217-ded2-4bcb-b82c-6520afc448a5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:06.917398 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d072217-ded2-4bcb-b82c-6520afc448a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:06.922887 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b75d9-6c46-49d1-b53f-ed75851f1dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:06.924782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b75d9-6c46-49d1-b53f-ed75851f1dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:06.925345 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b75d9-6c46-49d1-b53f-ed75851f1dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:06.925803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b75d9-6c46-49d1-b53f-ed75851f1dd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:06.935461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3d072217-ded2-4bcb-b82c-6520afc448a5 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:06.936806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3d072217-ded2-4bcb-b82c-6520afc448a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:06.939034 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b75d9-6c46-49d1-b53f-ed75851f1dd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:06.941531 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b75d9-6c46-49d1-b53f-ed75851f1dd6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:06.952916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:06.964781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1e5b75d9-6c46-49d1-b53f-ed75851f1dd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:06.964781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:06.964781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:06.964781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:06.964781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:06.964781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:06.964781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:06.964781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:06.964781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:06.964781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:06.964781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:06.964781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:06.964781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:06.964781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:06.964781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:06.966263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:06.966263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:06.966263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:06.966263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:06.966263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:06.966263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:06.966263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:06.966263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:06.966263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:06.966263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:06.966263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:06.966263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:06.966263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:06.966263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:06.966263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:06.966263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:06.966263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:06.967612 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:06.967612 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:06.967612 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:06.967612 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:06.971002 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 395/795] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:06.972055 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b75d9-6c46-49d1-b53f-ed75851f1dd6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:07.033061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 401/796] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:07.034278 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d072217-ded2-4bcb-b82c-6520afc448a5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:07.097014 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:07.131365 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b75d9-6c46-49d1-b53f-ed75851f1dd6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:07.147002 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d072217-ded2-4bcb-b82c-6520afc448a5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:07.176431 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f56ad-c389-424c-b25b-506fb659e144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:07.178543 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f56ad-c389-424c-b25b-506fb659e144 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:07.179034 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f56ad-c389-424c-b25b-506fb659e144 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:07.179661 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f56ad-c389-424c-b25b-506fb659e144 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:07.184896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ae3cb-a95b-4d3c-abca-67857979df48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:07.185569 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ae3cb-a95b-4d3c-abca-67857979df48 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:07.185852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ae3cb-a95b-4d3c-abca-67857979df48 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:07.186131 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6ae3cb-a95b-4d3c-abca-67857979df48 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:07.186909 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6ae3cb-a95b-4d3c-abca-67857979df48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:07.187273 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6ae3cb-a95b-4d3c-abca-67857979df48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:07.188695 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6ae3cb-a95b-4d3c-abca-67857979df48 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:07.195061 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f56ad-c389-424c-b25b-506fb659e144 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:07.197237 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f56ad-c389-424c-b25b-506fb659e144 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:07.207095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 402/797] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:07.207814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ae3cb-a95b-4d3c-abca-67857979df48 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:07.219978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-0c6f56ad-c389-424c-b25b-506fb659e144 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:07.219978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:07.219978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:07.219978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:07.219978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:07.219978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:07.219978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:07.219978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:07.219978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:07.219978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:07.219978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:07.219978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:07.219978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:07.219978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:07.219978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:07.220916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:07.220916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:07.220916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:07.220916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:07.220916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:07.220916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:07.220916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:07.220916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:07.220916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:07.220916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:07.220916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:07.220916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:07.220916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:07.220916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:07.220916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:07.220916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:07.220916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:07.222309 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:07.222309 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:07.222309 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:07.222309 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:07.229207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 396/798] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:07.229207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f56ad-c389-424c-b25b-506fb659e144 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:07.288302 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:07.339426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f56ad-c389-424c-b25b-506fb659e144 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:07.339685 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6ae3cb-a95b-4d3c-abca-67857979df48 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:07.357666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:07.378144 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61068e-65a2-4965-b3c0-3e625364a9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:07.378349 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be706980-7b75-4fbc-ad18-e852e09cc828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:07.378956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61068e-65a2-4965-b3c0-3e625364a9ba None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:07.379084 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be706980-7b75-4fbc-ad18-e852e09cc828 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:07.379241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61068e-65a2-4965-b3c0-3e625364a9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:07.379337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be706980-7b75-4fbc-ad18-e852e09cc828 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:07.379528 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61068e-65a2-4965-b3c0-3e625364a9ba None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:07.379628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be706980-7b75-4fbc-ad18-e852e09cc828 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:07.381285 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb61068e-65a2-4965-b3c0-3e625364a9ba None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:07.382668 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb61068e-65a2-4965-b3c0-3e625364a9ba None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:07.388461 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be706980-7b75-4fbc-ad18-e852e09cc828 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:07.389952 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be706980-7b75-4fbc-ad18-e852e09cc828 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:07.394824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-cb61068e-65a2-4965-b3c0-3e625364a9ba None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 16:14:07.394824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:07.394824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:14:07.394824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:14:07.394824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:14:07.394824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:14:07.394824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:14:07.394824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:07.394824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 16:14:07.394824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:07.394824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:14:07.394824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:07.394824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:07.394824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:07.394824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:07.394824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:07.396367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:07.396367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:07.396367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:07.396367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:07.396367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:07.396367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:07.396367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:07.396367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:14:07.396367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:14:07.396367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:14:07.396367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:14:07.396367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:07.396367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:07.396367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:07.396367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:07.396367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:07.397283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:07.397283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:07.397283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:07.397283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:07.397283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:07.397283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:07.397283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:07.397283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:07.397283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:07.397283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:07.397283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:07.397283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:14:07.397283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:07.397283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 16:14:07.397283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:07.397283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 403/799] 10.222.0.67 () {56 vars in 1129 bytes} [Thu Jun 30 16:14:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:07.398880 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61068e-65a2-4965-b3c0-3e625364a9ba None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:07.409943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-be706980-7b75-4fbc-ad18-e852e09cc828 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 16:14:07.409943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:07.409943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:07.409943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:07.409943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:07.409943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:07.409943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:07.409943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:07.409943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:07.409943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:07.409943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:07.409943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:07.409943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:14:07.409943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:14:07.409943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:14:07.409943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:14:07.410894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:07.410894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:07.410894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:14:07.410894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:07.410894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:14:07.410894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:07.410894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:07.410894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:07.410894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:07.410894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:07.410894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:07.410894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:07.410894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:07.410894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:07.410894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:07.410894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:07.410894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:07.410894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:07.412451 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:07.412451 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:07.412451 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:14:07.412451 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:14:07.412451 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:14:07.412451 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:14:07.412451 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:14:07.412451 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:14:07.412451 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 16:14:07.412451 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:07.413829 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 397/800] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:14:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:07.415678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71baae7-dbb7-4d46-a157-196b1309acff None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:07.416943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71baae7-dbb7-4d46-a157-196b1309acff None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:07.417385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71baae7-dbb7-4d46-a157-196b1309acff None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:07.422559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 398/801] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:07.422559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71baae7-dbb7-4d46-a157-196b1309acff None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:07.516761 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61068e-65a2-4965-b3c0-3e625364a9ba None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:07.529240 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71baae7-dbb7-4d46-a157-196b1309acff None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:07.544905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:07.552835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60800ecf-a895-430e-acd6-ed7b1934ceda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:07.553575 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60800ecf-a895-430e-acd6-ed7b1934ceda None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:07.553862 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60800ecf-a895-430e-acd6-ed7b1934ceda None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:07.554788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60800ecf-a895-430e-acd6-ed7b1934ceda None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:07.555021 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60800ecf-a895-430e-acd6-ed7b1934ceda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:07.555350 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60800ecf-a895-430e-acd6-ed7b1934ceda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:07.556768 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60800ecf-a895-430e-acd6-ed7b1934ceda None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:07.566890 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22176a-247f-4814-8baf-881a684d5bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:07.568176 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22176a-247f-4814-8baf-881a684d5bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:07.568459 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22176a-247f-4814-8baf-881a684d5bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:07.568732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22176a-247f-4814-8baf-881a684d5bd9 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:07.569434 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22176a-247f-4814-8baf-881a684d5bd9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:07.569770 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22176a-247f-4814-8baf-881a684d5bd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:07.574631 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 404/802] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:07.574631 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60800ecf-a895-430e-acd6-ed7b1934ceda None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:07.574883 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22176a-247f-4814-8baf-881a684d5bd9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:07.580660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 399/803] 10.222.0.67 () {56 vars in 1124 bytes} [Thu Jun 30 16:14:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:07.586524 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846fa35-32a9-49f6-8d5d-06fbc68fdd3a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:07.586819 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846fa35-32a9-49f6-8d5d-06fbc68fdd3a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:07.587104 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5846fa35-32a9-49f6-8d5d-06fbc68fdd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:07.605367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5846fa35-32a9-49f6-8d5d-06fbc68fdd3a None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:07.606580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5846fa35-32a9-49f6-8d5d-06fbc68fdd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:07.617798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:07.680048 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 400/804] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:07.681188 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846fa35-32a9-49f6-8d5d-06fbc68fdd3a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:07.691366 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60800ecf-a895-430e-acd6-ed7b1934ceda None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:07.727305 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ebe506-9975-4aba-a5ab-bbc62df5c2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:07.727992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebe506-9975-4aba-a5ab-bbc62df5c2e7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:07.728274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebe506-9975-4aba-a5ab-bbc62df5c2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:07.728554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebe506-9975-4aba-a5ab-bbc62df5c2e7 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:07.729376 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ebe506-9975-4aba-a5ab-bbc62df5c2e7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:07.729712 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ebe506-9975-4aba-a5ab-bbc62df5c2e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:07.731111 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ebe506-9975-4aba-a5ab-bbc62df5c2e7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:07.749447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 405/805] 10.222.0.67 () {56 vars in 1135 bytes} [Thu Jun 30 16:14:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:07.750238 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ebe506-9975-4aba-a5ab-bbc62df5c2e7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:07.793909 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5846fa35-32a9-49f6-8d5d-06fbc68fdd3a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:07.801413 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:07.827771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc1eb3d-cf21-4c5a-a971-8d3ca035a6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:07.828555 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1eb3d-cf21-4c5a-a971-8d3ca035a6bb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:07.828836 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1eb3d-cf21-4c5a-a971-8d3ca035a6bb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:07.829109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc1eb3d-cf21-4c5a-a971-8d3ca035a6bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:07.836448 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc1eb3d-cf21-4c5a-a971-8d3ca035a6bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:07.837921 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc1eb3d-cf21-4c5a-a971-8d3ca035a6bb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:07.845735 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:07.852102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-acc1eb3d-cf21-4c5a-a971-8d3ca035a6bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:07.852102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:07.852102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:07.852102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:07.852102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:07.852102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:07.852102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:07.852102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:07.852102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:07.852102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:07.852102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:07.852102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:07.852102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:14:07.852102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:14:07.852102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:14:07.853080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:14:07.853080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:07.853080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:07.853080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:07.853080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:07.853080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:07.853080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:07.853080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:07.853080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:07.853080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:07.853080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:07.853080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:07.853080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:07.853080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:07.853080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:07.853080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:07.853080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:07.853964 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:14:07.853964 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:14:07.853964 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:14:07.853964 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:14:07.853964 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:14:07.853964 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:14:07.853964 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:07.853964 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:07.854392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ebe506-9975-4aba-a5ab-bbc62df5c2e7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:07.854491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 401/806] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:14:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:07.855206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f247d3b-5072-4076-b9b9-a4a0a07856d7 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:07.855206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f247d3b-5072-4076-b9b9-a4a0a07856d7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:07.855408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f247d3b-5072-4076-b9b9-a4a0a07856d7 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:07.856720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 402/807] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:07.857331 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f247d3b-5072-4076-b9b9-a4a0a07856d7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:07.892825 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073af34-5d01-41a6-8e91-9bd8d1934af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:07.893409 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073af34-5d01-41a6-8e91-9bd8d1934af6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:07.893692 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073af34-5d01-41a6-8e91-9bd8d1934af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:07.893965 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073af34-5d01-41a6-8e91-9bd8d1934af6 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:07.894661 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073af34-5d01-41a6-8e91-9bd8d1934af6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:07.894965 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073af34-5d01-41a6-8e91-9bd8d1934af6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:07.897520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073af34-5d01-41a6-8e91-9bd8d1934af6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:07.902701 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 406/808] 10.222.0.67 () {56 vars in 1124 bytes} [Thu Jun 30 16:14:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:07.903484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073af34-5d01-41a6-8e91-9bd8d1934af6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:07.961031 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f247d3b-5072-4076-b9b9-a4a0a07856d7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:07.996139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42091f-1d0e-4411-b4ff-fc8ef3be4cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:07.996931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42091f-1d0e-4411-b4ff-fc8ef3be4cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:07.997214 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42091f-1d0e-4411-b4ff-fc8ef3be4cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:07.997487 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42091f-1d0e-4411-b4ff-fc8ef3be4cc3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:08.008335 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e42091f-1d0e-4411-b4ff-fc8ef3be4cc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:14:08.012360 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073af34-5d01-41a6-8e91-9bd8d1934af6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:08.016437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-6e42091f-1d0e-4411-b4ff-fc8ef3be4cc3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:14:08.016437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:08.016437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:08.016437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:08.016437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:08.016437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:08.016437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:08.016437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:08.016437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:08.016437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:08.016437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:08.016437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:08.016437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:14:08.016437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:14:08.016437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:14:08.016437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:14:08.017466 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:14:08.017466 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:14:08.017466 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:14:08.017466 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:14:08.017466 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:08.017466 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:08.017466 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:08.017466 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:08.017466 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:08.017466 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:08.017466 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:08.017466 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:08.017466 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:08.017466 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:08.017466 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:08.017466 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:08.018310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:08.018310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:08.018310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:08.018310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:08.018310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:14:08.018310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:14:08.018310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:14:08.018310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:14:08.018310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:14:08.018310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:14:08.018310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:14:08.018310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:08.018310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 403/809] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 30 16:14:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:08.019017 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42091f-1d0e-4411-b4ff-fc8ef3be4cc3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:08.047636 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:08.051763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e91225-3039-432f-a335-bb5b3a88e5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:08.052567 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e91225-3039-432f-a335-bb5b3a88e5e1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:08.052850 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e91225-3039-432f-a335-bb5b3a88e5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:08.053124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e91225-3039-432f-a335-bb5b3a88e5e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:08.063264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e91225-3039-432f-a335-bb5b3a88e5e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:08.064778 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e91225-3039-432f-a335-bb5b3a88e5e1 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:08.082391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-c9e91225-3039-432f-a335-bb5b3a88e5e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:08.082391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:08.082391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:08.082391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:08.082391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:08.082391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:08.082391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:08.082391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:08.082391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:08.082391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:08.082391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:08.082391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:08.082391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:08.082391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:08.082391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:08.084049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:08.084049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:08.084049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:08.084049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:08.084049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:08.084049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:08.084049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:08.084049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:08.084049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:08.084049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:08.084049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:08.084049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:08.084049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:08.084049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:08.084049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:08.084049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:08.084049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:08.084940 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:08.084940 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:08.084940 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:08.084940 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:08.084940 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 407/810] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:08.085469 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490c270-9e4e-4ca9-a20e-dfe70184fc72 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:08.085713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490c270-9e4e-4ca9-a20e-dfe70184fc72 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:08.085992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490c270-9e4e-4ca9-a20e-dfe70184fc72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:08.102553 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9490c270-9e4e-4ca9-a20e-dfe70184fc72 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:08.103860 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9490c270-9e4e-4ca9-a20e-dfe70184fc72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:08.111241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:08.127335 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42091f-1d0e-4411-b4ff-fc8ef3be4cc3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:08.162259 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48742975-7440-43cf-a510-ebe9cb95eb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:08.162840 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48742975-7440-43cf-a510-ebe9cb95eb1d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:08.163124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48742975-7440-43cf-a510-ebe9cb95eb1d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:08.163403 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48742975-7440-43cf-a510-ebe9cb95eb1d None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:08.164218 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48742975-7440-43cf-a510-ebe9cb95eb1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:08.164528 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48742975-7440-43cf-a510-ebe9cb95eb1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:08.165962 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48742975-7440-43cf-a510-ebe9cb95eb1d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:08.170962 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 408/811] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:08.171806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490c270-9e4e-4ca9-a20e-dfe70184fc72 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:08.184023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 404/812] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:14:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:08.184023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48742975-7440-43cf-a510-ebe9cb95eb1d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:08.269411 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:08.281424 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490c270-9e4e-4ca9-a20e-dfe70184fc72 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:08.284425 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48742975-7440-43cf-a510-ebe9cb95eb1d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:08.315666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131c715-493b-48f5-9b79-076d52836b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:08.316319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131c715-493b-48f5-9b79-076d52836b8f None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:08.316601 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131c715-493b-48f5-9b79-076d52836b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:08.316880 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0131c715-493b-48f5-9b79-076d52836b8f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:08.318264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131c715-493b-48f5-9b79-076d52836b8f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:08.319042 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaff594-10f3-4ac5-9bc7-f51d37d38326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:08.319817 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0131c715-493b-48f5-9b79-076d52836b8f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:08.319932 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaff594-10f3-4ac5-9bc7-f51d37d38326 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:08.320197 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaff594-10f3-4ac5-9bc7-f51d37d38326 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:08.320637 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaff594-10f3-4ac5-9bc7-f51d37d38326 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:08.321810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaff594-10f3-4ac5-9bc7-f51d37d38326 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:08.322114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaff594-10f3-4ac5-9bc7-f51d37d38326 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:08.323463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaff594-10f3-4ac5-9bc7-f51d37d38326 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:08.331895 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 405/813] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 30 16:14:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:08.332768 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaff594-10f3-4ac5-9bc7-f51d37d38326 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:08.417462 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:08.423321 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 409/814] 10.222.0.67 () {60 vars in 1144 bytes} [Thu Jun 30 16:14:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:14:08.424216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131c715-493b-48f5-9b79-076d52836b8f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:08.439539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaff594-10f3-4ac5-9bc7-f51d37d38326 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:08.485932 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce5187-0dc7-49e7-bb88-9bc6ab00e3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:08.487085 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce5187-0dc7-49e7-bb88-9bc6ab00e3c3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:08.487377 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce5187-0dc7-49e7-bb88-9bc6ab00e3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:08.487688 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce5187-0dc7-49e7-bb88-9bc6ab00e3c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:08.488417 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfce5187-0dc7-49e7-bb88-9bc6ab00e3c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:08.488726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfce5187-0dc7-49e7-bb88-9bc6ab00e3c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:08.490840 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfce5187-0dc7-49e7-bb88-9bc6ab00e3c3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:08.504047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 406/815] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:08.505346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce5187-0dc7-49e7-bb88-9bc6ab00e3c3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:08.509806 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:08.545895 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0131c715-493b-48f5-9b79-076d52836b8f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:08.586388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c9217-32f0-4b4c-a5e4-c392b97298df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:08.587057 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c9217-32f0-4b4c-a5e4-c392b97298df None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:08.587342 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c9217-32f0-4b4c-a5e4-c392b97298df None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:08.587647 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2c9217-32f0-4b4c-a5e4-c392b97298df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:08.588696 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c9217-32f0-4b4c-a5e4-c392b97298df None admin] RBAC: Adding query filter params (role.id=8acc34a163874a60b03728b814eaf075, scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d, user.id=71cca3708386459197abf6c67505bc5f) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:08.588894 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c9217-32f0-4b4c-a5e4-c392b97298df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:08.590376 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2c9217-32f0-4b4c-a5e4-c392b97298df None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:08.596666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 410/816] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:14:08 2022] GET /identity/v3/role_assignments?user.id=71cca3708386459197abf6c67505bc5f&scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d&role.id=8acc34a163874a60b03728b814eaf075 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:08.597705 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c9217-32f0-4b4c-a5e4-c392b97298df None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:08.623990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce5187-0dc7-49e7-bb88-9bc6ab00e3c3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:08.674257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae918bf-009b-4120-84de-c1ecc6bc4b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:08.674825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae918bf-009b-4120-84de-c1ecc6bc4b1f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:08.675015 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae918bf-009b-4120-84de-c1ecc6bc4b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:08.675285 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae918bf-009b-4120-84de-c1ecc6bc4b1f None admin] PATH_INFO: `/v3/services/network` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:08.677725 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae918bf-009b-4120-84de-c1ecc6bc4b1f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:08.678258 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae918bf-009b-4120-84de-c1ecc6bc4b1f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:08.690505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:08.692582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-cae918bf-009b-4120-84de-c1ecc6bc4b1f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 16:14:08.692582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:08.692582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:14:08.692582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:14:08.692582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:14:08.692582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:14:08.692582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:14:08.692582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:08.692582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 16:14:08.692582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:08.692582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:14:08.692582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:08.692582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:08.692582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:08.692582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:08.692582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:08.692582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:08.693648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:08.693648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:08.693648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:08.693648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:08.693648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:08.693648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:08.693648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:14:08.693648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:14:08.693648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:14:08.693648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:14:08.693648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:08.693648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:08.693648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:08.693648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:08.693648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:08.693648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:08.694506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:08.694506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:08.694506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:08.694506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:08.694506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:08.694506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:08.694506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:08.694506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:08.694506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:08.694506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:08.694506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:14:08.694506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:08.694506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 16:14:08.694506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:08.694506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 407/817] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:14:08 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:08.695308 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae918bf-009b-4120-84de-c1ecc6bc4b1f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:08.747712 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2c9217-32f0-4b4c-a5e4-c392b97298df None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:08.780044 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:08.794992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50026809-8cdd-4bea-9ad5-236f176226b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:08.796000 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50026809-8cdd-4bea-9ad5-236f176226b4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:08.796119 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50026809-8cdd-4bea-9ad5-236f176226b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:08.812879 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50026809-8cdd-4bea-9ad5-236f176226b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:08.812879 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50026809-8cdd-4bea-9ad5-236f176226b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:08.812879 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50026809-8cdd-4bea-9ad5-236f176226b4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:08.823411 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae918bf-009b-4120-84de-c1ecc6bc4b1f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:08.828961 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-50026809-8cdd-4bea-9ad5-236f176226b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:08.828961 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:08.828961 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:08.828961 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:08.828961 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:08.828961 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:08.828961 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:08.828961 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:08.828961 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:08.828961 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:08.828961 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:08.828961 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:08.828961 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:08.828961 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:08.828961 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:08.830017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:08.830017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:08.830017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:08.830017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:08.830017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:08.830017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:08.830017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:08.830017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:08.830017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:08.830017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:08.830017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:08.830017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:08.830017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:08.830017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:08.830017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:08.830017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:08.830017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:08.831185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:08.831185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:08.831185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:08.831185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:08.835586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 411/818] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:08.836698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50026809-8cdd-4bea-9ad5-236f176226b4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:08.867514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c889d80c-51e5-464d-a704-e62bd38f7f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:08.869063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889d80c-51e5-464d-a704-e62bd38f7f39 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:08.869063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889d80c-51e5-464d-a704-e62bd38f7f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:08.869285 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c889d80c-51e5-464d-a704-e62bd38f7f39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:08.880630 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c889d80c-51e5-464d-a704-e62bd38f7f39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:08.882092 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c889d80c-51e5-464d-a704-e62bd38f7f39 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:08.906328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-c889d80c-51e5-464d-a704-e62bd38f7f39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:08.906328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:08.906328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:08.906328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:08.906328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:08.906328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:08.906328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:08.906328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:08.906328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:08.906328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:08.906328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:08.906328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:08.906328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:14:08.906328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:14:08.906328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:14:08.907513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:14:08.907513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:08.907513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:08.907513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:08.907513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:08.907513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:08.907513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:08.907513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:08.907513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:08.907513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:08.907513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:08.907513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:08.907513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:08.907513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:08.907513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:08.907513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:08.907513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:08.908693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:14:08.908693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:14:08.908693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:14:08.908693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:14:08.908693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:14:08.908693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:14:08.908693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:08.908693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:08.912554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 408/819] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:14:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:08.913512 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c889d80c-51e5-464d-a704-e62bd38f7f39 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:08.974998 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50026809-8cdd-4bea-9ad5-236f176226b4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:08.989692 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:09.011995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd3b1b-6f2f-4d07-8052-ad596aeb65bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:09.012932 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd3b1b-6f2f-4d07-8052-ad596aeb65bb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:09.013394 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd3b1b-6f2f-4d07-8052-ad596aeb65bb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:09.013839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd3b1b-6f2f-4d07-8052-ad596aeb65bb None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:09.014699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cd3b1b-6f2f-4d07-8052-ad596aeb65bb None admin] RBAC: Adding query filter params (name=network) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:09.015180 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cd3b1b-6f2f-4d07-8052-ad596aeb65bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:09.016777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cd3b1b-6f2f-4d07-8052-ad596aeb65bb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:09.022839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 412/820] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 30 16:14:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:09.024238 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd3b1b-6f2f-4d07-8052-ad596aeb65bb None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:09.052944 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c889d80c-51e5-464d-a704-e62bd38f7f39 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:09.086036 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:09.094079 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49de04-5b15-4ab7-87c2-06051bdb1d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:09.095146 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49de04-5b15-4ab7-87c2-06051bdb1d50 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:09.095146 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49de04-5b15-4ab7-87c2-06051bdb1d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:09.099041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49de04-5b15-4ab7-87c2-06051bdb1d50 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:09.099041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49de04-5b15-4ab7-87c2-06051bdb1d50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:09.099041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49de04-5b15-4ab7-87c2-06051bdb1d50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:09.099041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49de04-5b15-4ab7-87c2-06051bdb1d50 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:09.115362 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 409/821] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:09.115362 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49de04-5b15-4ab7-87c2-06051bdb1d50 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:09.144199 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd3b1b-6f2f-4d07-8052-ad596aeb65bb None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:09.189976 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030f01b-a9b6-4509-b320-002ef981b81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:09.190737 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030f01b-a9b6-4509-b320-002ef981b81e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:09.191027 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030f01b-a9b6-4509-b320-002ef981b81e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:09.191308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030f01b-a9b6-4509-b320-002ef981b81e None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:09.192675 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030f01b-a9b6-4509-b320-002ef981b81e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:09.192988 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030f01b-a9b6-4509-b320-002ef981b81e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:09.194661 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030f01b-a9b6-4509-b320-002ef981b81e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:09.208925 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 413/822] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:14:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:09.209995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030f01b-a9b6-4509-b320-002ef981b81e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:09.234930 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49de04-5b15-4ab7-87c2-06051bdb1d50 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:09.255688 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:09.272081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26684332-50d9-43ae-8a98-23bed8cf0da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:09.272761 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26684332-50d9-43ae-8a98-23bed8cf0da6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:09.273311 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26684332-50d9-43ae-8a98-23bed8cf0da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:09.273591 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26684332-50d9-43ae-8a98-23bed8cf0da6 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:09.274310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26684332-50d9-43ae-8a98-23bed8cf0da6 None admin] RBAC: Adding query filter params (type=network) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:09.274620 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26684332-50d9-43ae-8a98-23bed8cf0da6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:09.276025 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26684332-50d9-43ae-8a98-23bed8cf0da6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:09.281555 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 410/823] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 30 16:14:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:09.282355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26684332-50d9-43ae-8a98-23bed8cf0da6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:09.325223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030f01b-a9b6-4509-b320-002ef981b81e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:09.341018 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:09.364157 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b34372-277f-4728-991b-7b0bcfb6a66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:09.365248 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b34372-277f-4728-991b-7b0bcfb6a66b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:09.365857 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b34372-277f-4728-991b-7b0bcfb6a66b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:09.366295 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b34372-277f-4728-991b-7b0bcfb6a66b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:09.374159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b34372-277f-4728-991b-7b0bcfb6a66b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:09.375956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b34372-277f-4728-991b-7b0bcfb6a66b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:09.392617 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26684332-50d9-43ae-8a98-23bed8cf0da6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:09.393697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-b1b34372-277f-4728-991b-7b0bcfb6a66b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 16:14:09.393697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:09.393697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:09.393697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:09.393697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:09.393697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:09.393697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:09.393697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:09.393697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:09.393697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:09.393697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:09.393697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:09.393697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:14:09.393697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:14:09.393697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:14:09.393697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:14:09.394814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:09.394814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:09.394814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:14:09.394814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:09.394814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:14:09.394814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:09.394814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:09.394814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:09.394814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:09.394814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:09.394814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:09.394814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:09.394814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:09.394814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:09.394814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:09.394814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:09.394814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:09.394814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:09.396185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:09.396185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:09.396185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:14:09.396185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:14:09.396185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:14:09.396185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:14:09.396185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:14:09.396185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:14:09.396185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 16:14:09.396185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:09.398696 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 414/824] 10.222.0.67 () {56 vars in 1117 bytes} [Thu Jun 30 16:14:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:09.399777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b34372-277f-4728-991b-7b0bcfb6a66b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:09.431024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894f297-a002-450c-af50-c5b6a146787e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:09.432127 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894f297-a002-450c-af50-c5b6a146787e None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:09.432343 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894f297-a002-450c-af50-c5b6a146787e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:09.432605 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894f297-a002-450c-af50-c5b6a146787e None admin] PATH_INFO: `/v3/projects/19ef92ebcce94b90bf39ad3a58772a1d/users/988b917eea4b48ffae32e23b8555a5c6/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:09.434722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2894f297-a002-450c-af50-c5b6a146787e None admin] RBAC: Authorizing `identity:create_grant(project_id=19ef92ebcce94b90bf39ad3a58772a1d, user_id=988b917eea4b48ffae32e23b8555a5c6, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:09.436332 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2894f297-a002-450c-af50-c5b6a146787e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:09.505772 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:09.509805 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b34372-277f-4728-991b-7b0bcfb6a66b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:09.542856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 411/825] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 30 16:14:09 2022] PUT /identity/v3/projects/19ef92ebcce94b90bf39ad3a58772a1d/users/988b917eea4b48ffae32e23b8555a5c6/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:14:09.544263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894f297-a002-450c-af50-c5b6a146787e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:09.570080 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d89376-7a8e-4991-b6a0-a0e3638f1150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:09.570799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d89376-7a8e-4991-b6a0-a0e3638f1150 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:09.571085 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d89376-7a8e-4991-b6a0-a0e3638f1150 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:09.571359 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d89376-7a8e-4991-b6a0-a0e3638f1150 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:09.572152 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d89376-7a8e-4991-b6a0-a0e3638f1150 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:09.576201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d89376-7a8e-4991-b6a0-a0e3638f1150 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:09.693168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 415/826] 10.222.0.67 () {60 vars in 1144 bytes} [Thu Jun 30 16:14:09 2022] POST /identity/v3/endpoints => generated 327 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:14:09.704069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2894f297-a002-450c-af50-c5b6a146787e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:09.721958 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:09.758616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f567a-3f43-4280-b454-b1ae4a89321a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:09.759796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f567a-3f43-4280-b454-b1ae4a89321a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:09.760271 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f567a-3f43-4280-b454-b1ae4a89321a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:09.760765 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f567a-3f43-4280-b454-b1ae4a89321a None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:09.761860 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f567a-3f43-4280-b454-b1ae4a89321a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:09.762812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f567a-3f43-4280-b454-b1ae4a89321a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:09.764659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f567a-3f43-4280-b454-b1ae4a89321a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:09.789144 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 412/827] 10.222.0.67 () {56 vars in 1149 bytes} [Thu Jun 30 16:14:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:09.795345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d89376-7a8e-4991-b6a0-a0e3638f1150 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:09.922252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d89376-7a8e-4991-b6a0-a0e3638f1150 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:09.963620 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b6b0f-4a76-4d62-b504-20e880f0c530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:09.964848 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b6b0f-4a76-4d62-b504-20e880f0c530 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:09.965339 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b6b0f-4a76-4d62-b504-20e880f0c530 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:09.965895 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b6b0f-4a76-4d62-b504-20e880f0c530 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:09.978949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b6b0f-4a76-4d62-b504-20e880f0c530 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:09.980768 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b6b0f-4a76-4d62-b504-20e880f0c530 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:09.999789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-954b6b0f-4a76-4d62-b504-20e880f0c530 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:09.999789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:09.999789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:09.999789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:09.999789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:09.999789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:09.999789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:09.999789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:09.999789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:09.999789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:09.999789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:09.999789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:09.999789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:09.999789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:09.999789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:10.000713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:10.000713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:10.000713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:10.000713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:10.000713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:10.000713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:10.000713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:10.000713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:10.000713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:10.000713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:10.000713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:10.000713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:10.000713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:10.000713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:10.000713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:10.000713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:10.000713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:10.002097 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:10.002097 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:10.002097 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:10.002097 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:10.004497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 416/828] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:10.010203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f567a-3f43-4280-b454-b1ae4a89321a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:10.119915 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f567a-3f43-4280-b454-b1ae4a89321a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:10.157443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd253b3-3479-4943-a595-167985c503d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:10.158577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd253b3-3479-4943-a595-167985c503d5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:10.159069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd253b3-3479-4943-a595-167985c503d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:10.159610 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd253b3-3479-4943-a595-167985c503d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:10.160536 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd253b3-3479-4943-a595-167985c503d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:10.161863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd253b3-3479-4943-a595-167985c503d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:10.163513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd253b3-3479-4943-a595-167985c503d5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:10.177071 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 413/829] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:10.182903 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b6b0f-4a76-4d62-b504-20e880f0c530 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:10.214668 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:10.303142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b6b0f-4a76-4d62-b504-20e880f0c530 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:10.345226 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9f8ba-f06a-4970-bd98-4481cd1200f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:10.346826 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9f8ba-f06a-4970-bd98-4481cd1200f7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:10.347320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9f8ba-f06a-4970-bd98-4481cd1200f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:10.347838 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9f8ba-f06a-4970-bd98-4481cd1200f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:10.358910 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d9f8ba-f06a-4970-bd98-4481cd1200f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:10.360846 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d9f8ba-f06a-4970-bd98-4481cd1200f7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:10.377133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f4d9f8ba-f06a-4970-bd98-4481cd1200f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:10.377133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:10.377133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:10.377133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:10.377133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:10.377133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:10.377133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:10.377133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:10.377133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:10.377133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:10.377133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:10.377133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:10.377133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:14:10.377133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:14:10.377133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:14:10.378195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:14:10.378195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:10.378195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:10.378195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:10.378195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:10.378195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:10.378195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:10.378195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:10.378195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:10.378195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:10.378195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:10.378195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:10.378195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:10.378195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:10.378195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:10.378195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:10.378195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:10.379108 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:14:10.379108 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:14:10.379108 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:14:10.379108 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:14:10.379108 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:14:10.379108 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:14:10.379108 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:10.379108 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:10.381927 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 417/830] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:14:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:10.388604 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd253b3-3479-4943-a595-167985c503d5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:10.420026 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:10.497706 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd253b3-3479-4943-a595-167985c503d5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:10.533794 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905b50a-93f7-4fe2-8b05-27ff2d78d757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:10.534525 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905b50a-93f7-4fe2-8b05-27ff2d78d757 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:10.534812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905b50a-93f7-4fe2-8b05-27ff2d78d757 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:10.535087 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905b50a-93f7-4fe2-8b05-27ff2d78d757 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:10.536625 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905b50a-93f7-4fe2-8b05-27ff2d78d757 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:10.536997 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905b50a-93f7-4fe2-8b05-27ff2d78d757 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:10.541536 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905b50a-93f7-4fe2-8b05-27ff2d78d757 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:10.551612 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 414/831] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:14:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:10.557444 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9f8ba-f06a-4970-bd98-4481cd1200f7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:10.628766 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:10.666853 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9f8ba-f06a-4970-bd98-4481cd1200f7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:10.700995 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73679548-aabc-4cbb-afb0-2278440bf3a9 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:10.701377 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73679548-aabc-4cbb-afb0-2278440bf3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:10.701584 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73679548-aabc-4cbb-afb0-2278440bf3a9 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:10.704236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb366baa-15d3-4525-b8c4-8fce63605953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:10.704368 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 415/832] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:10.705469 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb366baa-15d3-4525-b8c4-8fce63605953 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:10.705963 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb366baa-15d3-4525-b8c4-8fce63605953 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:10.706443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb366baa-15d3-4525-b8c4-8fce63605953 None admin] PATH_INFO: `/v3/projects/19ef92ebcce94b90bf39ad3a58772a1d/users/d49604bffa3d4e89a631eefa6d4e4f5d/roles/57bc5df632b3407d97116ae34d92a5cb` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:10.708780 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950a91b-b9bb-488c-9e22-4402c60bc921 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:10.709067 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950a91b-b9bb-488c-9e22-4402c60bc921 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:10.709351 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a950a91b-b9bb-488c-9e22-4402c60bc921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:10.725624 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb366baa-15d3-4525-b8c4-8fce63605953 None admin] RBAC: Authorizing `identity:create_grant(project_id=19ef92ebcce94b90bf39ad3a58772a1d, user_id=d49604bffa3d4e89a631eefa6d4e4f5d, role_id=57bc5df632b3407d97116ae34d92a5cb)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:10.726435 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a950a91b-b9bb-488c-9e22-4402c60bc921 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:10.727672 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a950a91b-b9bb-488c-9e22-4402c60bc921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:10.728015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb366baa-15d3-4525-b8c4-8fce63605953 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:10.803321 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 418/834] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 30 16:14:10 2022] PUT /identity/v3/projects/19ef92ebcce94b90bf39ad3a58772a1d/users/d49604bffa3d4e89a631eefa6d4e4f5d/roles/57bc5df632b3407d97116ae34d92a5cb => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:14:10.807656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 416/834] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 16:14:10.825383 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb366baa-15d3-4525-b8c4-8fce63605953 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:10.947545 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2ceba-fa65-4df3-a945-1fc253b5fcf1 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:10.948210 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2ceba-fa65-4df3-a945-1fc253b5fcf1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:10.948749 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e2ceba-fa65-4df3-a945-1fc253b5fcf1 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:10.954931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 417/835] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:10.957261 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bd4e6-5bb8-4db7-b4da-7653278d31c4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:10.958494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bd4e6-5bb8-4db7-b4da-7653278d31c4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:10.959241 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bd4e6-5bb8-4db7-b4da-7653278d31c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:10.959621 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb366baa-15d3-4525-b8c4-8fce63605953 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:10.981102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-718bd4e6-5bb8-4db7-b4da-7653278d31c4 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:10.982462 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-718bd4e6-5bb8-4db7-b4da-7653278d31c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:10.991167 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:10.994523 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:11.013829 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45082754-9381-488a-aeb0-3ba3527b1cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:11.014848 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45082754-9381-488a-aeb0-3ba3527b1cbb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:11.015154 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45082754-9381-488a-aeb0-3ba3527b1cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:11.015398 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45082754-9381-488a-aeb0-3ba3527b1cbb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:11.016136 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45082754-9381-488a-aeb0-3ba3527b1cbb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:11.017512 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45082754-9381-488a-aeb0-3ba3527b1cbb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:11.038143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-45082754-9381-488a-aeb0-3ba3527b1cbb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 16:14:11.038143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:11.038143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:14:11.038143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:14:11.038143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:14:11.038143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:14:11.038143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:14:11.038143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:11.038143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 16:14:11.038143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:11.038143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:14:11.038143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:11.038143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:11.038143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:11.038143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:11.038143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:11.038143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:11.039210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:11.039210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:11.039210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:11.039210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:11.039210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:11.039210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:11.039210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:14:11.039210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:14:11.039210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:14:11.039210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:14:11.039210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:11.039210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:11.039210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:11.039210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:11.039210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:11.039210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:11.040492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:11.040492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:11.040492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:11.040492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:11.040492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:11.040492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:11.040492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:11.040492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:11.040492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:11.040492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:11.040492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:14:11.040492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:11.040492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 16:14:11.040492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:11.040492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 419/836] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 30 16:14:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:11.045330 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45082754-9381-488a-aeb0-3ba3527b1cbb None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:11.091541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 418/837] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:10 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:11.107582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718bd4e6-5bb8-4db7-b4da-7653278d31c4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:11.160189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45082754-9381-488a-aeb0-3ba3527b1cbb None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:11.197760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec90cb-9c10-4c7a-b3d1-ec6eee7dc068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:11.198426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec90cb-9c10-4c7a-b3d1-ec6eee7dc068 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:11.198714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec90cb-9c10-4c7a-b3d1-ec6eee7dc068 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:11.198989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec90cb-9c10-4c7a-b3d1-ec6eee7dc068 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:11.199744 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec90cb-9c10-4c7a-b3d1-ec6eee7dc068 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:11.200023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec90cb-9c10-4c7a-b3d1-ec6eee7dc068 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:11.201393 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec90cb-9c10-4c7a-b3d1-ec6eee7dc068 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:11.206655 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 420/838] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 30 16:14:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:11.213342 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec90cb-9c10-4c7a-b3d1-ec6eee7dc068 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:11.218621 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718bd4e6-5bb8-4db7-b4da-7653278d31c4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:11.255817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfdc261-b811-4288-8428-8a6a289e690a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:11.257007 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfdc261-b811-4288-8428-8a6a289e690a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:11.257494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfdc261-b811-4288-8428-8a6a289e690a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:11.258028 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfdc261-b811-4288-8428-8a6a289e690a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:11.259025 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfdc261-b811-4288-8428-8a6a289e690a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:11.260629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfdc261-b811-4288-8428-8a6a289e690a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:11.265757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:11.275086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9bfdc261-b811-4288-8428-8a6a289e690a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 16:14:11.275086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:11.275086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:14:11.275086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:14:11.275086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:14:11.275086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:14:11.275086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:14:11.275086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:11.275086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 16:14:11.275086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:11.275086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:14:11.275086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:11.275086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:11.275086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:11.275086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:11.275086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:11.284693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:11.284693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:11.284693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:11.284693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:11.284693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:11.284693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:11.284693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:11.284693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:14:11.284693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:14:11.284693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:14:11.284693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:14:11.284693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:11.284693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:11.284693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:11.284693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:11.284693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:11.286296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:11.286296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:11.286296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:11.286296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:11.286296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:11.286296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:11.286296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:11.286296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:11.286296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:11.286296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:11.286296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:11.286296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:14:11.286296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:11.286296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 16:14:11.286296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:11.286296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 419/839] 10.222.0.67 () {56 vars in 1132 bytes} [Thu Jun 30 16:14:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:14:11.288017 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfdc261-b811-4288-8428-8a6a289e690a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:11.299866 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:11.345968 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec90cb-9c10-4c7a-b3d1-ec6eee7dc068 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:11.391703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fcdec1-50b7-4150-87ed-8dd9145e8637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:11.392803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fcdec1-50b7-4150-87ed-8dd9145e8637 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:11.393188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fcdec1-50b7-4150-87ed-8dd9145e8637 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:11.393568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fcdec1-50b7-4150-87ed-8dd9145e8637 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:11.394555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fcdec1-50b7-4150-87ed-8dd9145e8637 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:11.394966 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fcdec1-50b7-4150-87ed-8dd9145e8637 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:11.397768 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fcdec1-50b7-4150-87ed-8dd9145e8637 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:11.403071 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfdc261-b811-4288-8428-8a6a289e690a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:11.415797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 421/840] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 30 16:14:11 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:11.446054 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d407ab-3bf2-4322-b840-44ec48a2d9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:11.449729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d407ab-3bf2-4322-b840-44ec48a2d9c4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:11.449729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d407ab-3bf2-4322-b840-44ec48a2d9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:11.449729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d407ab-3bf2-4322-b840-44ec48a2d9c4 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:11.449729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d407ab-3bf2-4322-b840-44ec48a2d9c4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:11.449729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d407ab-3bf2-4322-b840-44ec48a2d9c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:11.450248 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d407ab-3bf2-4322-b840-44ec48a2d9c4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:11.461328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 420/841] 10.222.0.67 () {56 vars in 1126 bytes} [Thu Jun 30 16:14:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:11.462870 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fcdec1-50b7-4150-87ed-8dd9145e8637 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:11.584867 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fcdec1-50b7-4150-87ed-8dd9145e8637 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:11.595376 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:11.622346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b96a59-bc63-499e-a003-53956238a54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:11.622937 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b96a59-bc63-499e-a003-53956238a54c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:11.623224 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b96a59-bc63-499e-a003-53956238a54c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:11.623533 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b96a59-bc63-499e-a003-53956238a54c None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:11.624253 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b96a59-bc63-499e-a003-53956238a54c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:11.624559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b96a59-bc63-499e-a003-53956238a54c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:11.625943 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b96a59-bc63-499e-a003-53956238a54c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:11.640026 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 422/842] 10.222.0.67 () {56 vars in 1126 bytes} [Thu Jun 30 16:14:11 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:11.873730 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241755c1-2ea6-4b64-8ba5-c2addea70540 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:11.873922 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241755c1-2ea6-4b64-8ba5-c2addea70540 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:11.874194 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241755c1-2ea6-4b64-8ba5-c2addea70540 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:11.875710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 421/843] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:11.882228 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571f8e7-b9eb-4020-84f2-9f5ef4f4283d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:11.882593 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571f8e7-b9eb-4020-84f2-9f5ef4f4283d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:11.882877 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571f8e7-b9eb-4020-84f2-9f5ef4f4283d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:11.900707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a571f8e7-b9eb-4020-84f2-9f5ef4f4283d None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:11.901967 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a571f8e7-b9eb-4020-84f2-9f5ef4f4283d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:11.978072 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 423/844] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:11 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:12.002045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241755c1-2ea6-4b64-8ba5-c2addea70540 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:12.088658 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:12.116632 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241755c1-2ea6-4b64-8ba5-c2addea70540 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:12.153199 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949c80c-bc69-4fe0-a238-4c6e6f2c9d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:12.153921 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949c80c-bc69-4fe0-a238-4c6e6f2c9d45 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:12.154201 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949c80c-bc69-4fe0-a238-4c6e6f2c9d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:12.154477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949c80c-bc69-4fe0-a238-4c6e6f2c9d45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:12.166012 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2949c80c-bc69-4fe0-a238-4c6e6f2c9d45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:14:12.173957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-2949c80c-bc69-4fe0-a238-4c6e6f2c9d45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:14:12.173957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:12.173957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:12.173957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:12.173957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:12.173957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:12.173957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:12.173957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:12.173957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:12.173957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:12.173957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:12.173957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:12.173957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:14:12.173957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:14:12.173957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:14:12.173957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:14:12.174927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:14:12.174927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:14:12.174927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:14:12.174927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:14:12.174927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:12.174927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:12.174927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:12.174927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:12.174927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:12.174927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:12.174927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:12.174927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:12.174927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:12.174927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:12.174927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:12.174927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:12.175781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:12.175781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:12.175781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:12.175781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:12.175781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:14:12.175781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:14:12.175781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:14:12.175781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:14:12.175781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:14:12.175781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:14:12.175781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:14:12.175781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:12.175781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 422/845] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:14:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:12.182209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a571f8e7-b9eb-4020-84f2-9f5ef4f4283d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:12.286754 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a571f8e7-b9eb-4020-84f2-9f5ef4f4283d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:12.288403 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:12.321460 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38244b7-72d1-45fc-a0b2-2ca8de1f5b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:12.322128 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38244b7-72d1-45fc-a0b2-2ca8de1f5b7c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:12.322407 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38244b7-72d1-45fc-a0b2-2ca8de1f5b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:12.322682 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38244b7-72d1-45fc-a0b2-2ca8de1f5b7c None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:12.323481 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38244b7-72d1-45fc-a0b2-2ca8de1f5b7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:12.323781 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38244b7-72d1-45fc-a0b2-2ca8de1f5b7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:12.325368 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38244b7-72d1-45fc-a0b2-2ca8de1f5b7c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:12.334932 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 424/846] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:14:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:12.340214 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949c80c-bc69-4fe0-a238-4c6e6f2c9d45 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:12.445852 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949c80c-bc69-4fe0-a238-4c6e6f2c9d45 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:12.480532 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df793c1-6abb-4b7a-abeb-c8bda2604c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:12.481404 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df793c1-6abb-4b7a-abeb-c8bda2604c3e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:12.481682 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df793c1-6abb-4b7a-abeb-c8bda2604c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:12.481953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df793c1-6abb-4b7a-abeb-c8bda2604c3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:12.489824 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:12.493627 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df793c1-6abb-4b7a-abeb-c8bda2604c3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:12.495083 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df793c1-6abb-4b7a-abeb-c8bda2604c3e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:12.512705 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-3df793c1-6abb-4b7a-abeb-c8bda2604c3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:12.512705 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:12.512705 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:12.512705 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:12.512705 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:12.512705 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:12.512705 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:12.512705 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:12.512705 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:12.512705 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:12.512705 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:12.512705 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:12.512705 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:12.512705 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:12.512705 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:12.514723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:12.514723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:12.514723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:12.514723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:12.514723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:12.514723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:12.514723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:12.514723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:12.514723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:12.514723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:12.514723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:12.514723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:12.514723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:12.514723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:12.514723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:12.514723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:12.514723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:12.516088 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:12.516088 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:12.516088 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:12.516088 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:12.516088 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 423/847] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:14:12.519537 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38244b7-72d1-45fc-a0b2-2ca8de1f5b7c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:12.625535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38244b7-72d1-45fc-a0b2-2ca8de1f5b7c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:12.663438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4f94a-4717-4b45-9b2b-635512e61ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:12.664130 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4f94a-4717-4b45-9b2b-635512e61ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:12.664412 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4f94a-4717-4b45-9b2b-635512e61ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:12.664688 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4f94a-4717-4b45-9b2b-635512e61ed8 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:12.665408 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4f94a-4717-4b45-9b2b-635512e61ed8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:12.665804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4f94a-4717-4b45-9b2b-635512e61ed8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:12.667182 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4f94a-4717-4b45-9b2b-635512e61ed8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:12.681191 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 425/848] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:12.688656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df793c1-6abb-4b7a-abeb-c8bda2604c3e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:12.799736 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df793c1-6abb-4b7a-abeb-c8bda2604c3e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:12.837791 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa4d033-5985-4667-b157-9a58b531a0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:12.838669 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4d033-5985-4667-b157-9a58b531a0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:12.838950 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4d033-5985-4667-b157-9a58b531a0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:12.839218 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4d033-5985-4667-b157-9a58b531a0e7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:12.847307 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa4d033-5985-4667-b157-9a58b531a0e7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:12.850062 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa4d033-5985-4667-b157-9a58b531a0e7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:12.864756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-7aa4d033-5985-4667-b157-9a58b531a0e7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 16:14:12.864756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:12.864756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:12.864756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:12.864756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:12.864756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:12.864756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:12.864756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:12.864756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:12.864756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:12.864756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:12.864756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:12.864756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:14:12.864756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:14:12.864756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:14:12.864756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:14:12.865723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:12.865723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:12.865723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:14:12.865723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:12.865723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:14:12.865723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:12.865723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:12.865723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:12.865723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:12.865723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:12.865723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:12.865723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:12.865723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:12.865723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:12.865723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:12.865723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:12.865723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:12.865723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:12.866690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:12.866690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:12.866690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:14:12.866690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:14:12.866690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:14:12.866690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:14:12.866690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:14:12.866690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:14:12.866690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 16:14:12.866690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:12.866690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 424/849] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:14:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:12.876679 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4f94a-4717-4b45-9b2b-635512e61ed8 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:12.903833 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:12.982786 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef810cb-365d-40e3-8288-effcf05ba59e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:12.983117 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef810cb-365d-40e3-8288-effcf05ba59e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:12.983374 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef810cb-365d-40e3-8288-effcf05ba59e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:12.985280 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 425/850] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:14:12.985501 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4f94a-4717-4b45-9b2b-635512e61ed8 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:12.992951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a98f1-1c18-4dcc-8363-025e64f6274e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:12.993532 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a98f1-1c18-4dcc-8363-025e64f6274e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:12.994011 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a98f1-1c18-4dcc-8363-025e64f6274e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:13.013039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f77a98f1-1c18-4dcc-8363-025e64f6274e None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:13.014770 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f77a98f1-1c18-4dcc-8363-025e64f6274e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:13.033945 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231ee86-1ff3-4f5a-a8fa-e0aada50e9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:13.034620 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231ee86-1ff3-4f5a-a8fa-e0aada50e9d6 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:13.034932 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231ee86-1ff3-4f5a-a8fa-e0aada50e9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:13.035217 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231ee86-1ff3-4f5a-a8fa-e0aada50e9d6 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:13.036483 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231ee86-1ff3-4f5a-a8fa-e0aada50e9d6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:13.036910 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231ee86-1ff3-4f5a-a8fa-e0aada50e9d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:13.038447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231ee86-1ff3-4f5a-a8fa-e0aada50e9d6 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:13.062841 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 426/851] 10.222.0.67 () {56 vars in 1135 bytes} [Thu Jun 30 16:14:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:13.064392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231ee86-1ff3-4f5a-a8fa-e0aada50e9d6 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:13.068801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:13.111494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 426/852] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:12 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 16:14:13.129003 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a98f1-1c18-4dcc-8363-025e64f6274e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:13.163372 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:13.182915 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231ee86-1ff3-4f5a-a8fa-e0aada50e9d6 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:13.221869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a1d83-6d8f-4b23-b440-6c1932b6df7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:13.222826 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a1d83-6d8f-4b23-b440-6c1932b6df7d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:13.223351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a1d83-6d8f-4b23-b440-6c1932b6df7d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:13.224037 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a1d83-6d8f-4b23-b440-6c1932b6df7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:13.237229 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a98f1-1c18-4dcc-8363-025e64f6274e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:13.274377 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14a53d-e7a5-4d61-9ee5-dd5e7eab2757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:13.275348 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14a53d-e7a5-4d61-9ee5-dd5e7eab2757 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:13.275746 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14a53d-e7a5-4d61-9ee5-dd5e7eab2757 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:13.276118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14a53d-e7a5-4d61-9ee5-dd5e7eab2757 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:13.287611 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b14a53d-e7a5-4d61-9ee5-dd5e7eab2757 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:14:13.295863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1b14a53d-e7a5-4d61-9ee5-dd5e7eab2757 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:14:13.295863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:13.295863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:13.295863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:13.295863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:13.295863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:13.295863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:13.295863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:13.295863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:13.295863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:13.295863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:13.295863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:13.295863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:14:13.295863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:14:13.295863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:14:13.295863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:14:13.296817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:14:13.296817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:14:13.296817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:14:13.296817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:14:13.296817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:13.296817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:13.296817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:13.296817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:13.296817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:13.296817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:13.296817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:13.296817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:13.296817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:13.296817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:13.296817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:13.296817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:13.297647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:13.297647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:13.297647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:13.297647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:13.297647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:14:13.297647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:14:13.297647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:14:13.297647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:14:13.297647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:14:13.297647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:14:13.297647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:14:13.297647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:13.300600 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a1d83-6d8f-4b23-b440-6c1932b6df7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:13.302946 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a1d83-6d8f-4b23-b440-6c1932b6df7d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:13.305610 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 427/853] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 30 16:14:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:14:13.306656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14a53d-e7a5-4d61-9ee5-dd5e7eab2757 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:13.344017 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:13.392833 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:13.414740 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14a53d-e7a5-4d61-9ee5-dd5e7eab2757 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:13.415800 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 427/854] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:14:13 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 16:14:13.425568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a1d83-6d8f-4b23-b440-6c1932b6df7d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:13.455528 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277bfdf-678b-4272-a303-b2ae5ad96a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:13.456342 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277bfdf-678b-4272-a303-b2ae5ad96a40 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:13.456743 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277bfdf-678b-4272-a303-b2ae5ad96a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:13.457122 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277bfdf-678b-4272-a303-b2ae5ad96a40 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:13.458494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277bfdf-678b-4272-a303-b2ae5ad96a40 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:13.458899 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277bfdf-678b-4272-a303-b2ae5ad96a40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:13.460441 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277bfdf-678b-4272-a303-b2ae5ad96a40 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:13.469363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 428/855] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 30 16:14:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:13.475420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277bfdf-678b-4272-a303-b2ae5ad96a40 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:13.544558 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a1d83-6d8f-4b23-b440-6c1932b6df7d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:13.579683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:13.583889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277bfdf-678b-4272-a303-b2ae5ad96a40 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:13.592091 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e902c95-df82-49ef-8e79-8f0a8c17b323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:13.593132 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e902c95-df82-49ef-8e79-8f0a8c17b323 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:13.593611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e902c95-df82-49ef-8e79-8f0a8c17b323 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:13.594100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e902c95-df82-49ef-8e79-8f0a8c17b323 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:13.606249 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e902c95-df82-49ef-8e79-8f0a8c17b323 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:13.608178 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e902c95-df82-49ef-8e79-8f0a8c17b323 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:13.620588 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a5778-94b8-4006-8aec-2fb6f4b1427b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:13.621394 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a5778-94b8-4006-8aec-2fb6f4b1427b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:13.621669 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a5778-94b8-4006-8aec-2fb6f4b1427b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:13.621944 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a5778-94b8-4006-8aec-2fb6f4b1427b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:13.626989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8e902c95-df82-49ef-8e79-8f0a8c17b323 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:13.626989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:13.626989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:13.626989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:13.626989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:13.626989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:13.626989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:13.626989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:13.626989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:13.626989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:13.626989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:13.626989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:13.626989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:13.626989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:13.626989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:13.628019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:13.628019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:13.628019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:13.628019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:13.628019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:13.628019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:13.628019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:13.628019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:13.628019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:13.628019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:13.628019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:13.628019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:13.628019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:13.628019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:13.628019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:13.628019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:13.628019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:13.628999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:13.628999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:13.628999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:13.628999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:13.631811 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 428/856] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:13.632895 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a5778-94b8-4006-8aec-2fb6f4b1427b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:13.633751 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f91833-45ec-45bf-8b5e-753309c3b688 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:13.634048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f91833-45ec-45bf-8b5e-753309c3b688 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:13.634333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a5778-94b8-4006-8aec-2fb6f4b1427b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:13.634442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f91833-45ec-45bf-8b5e-753309c3b688 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:13.635837 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 429/857] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:13.640438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f91833-45ec-45bf-8b5e-753309c3b688 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:13.653833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-b13a5778-94b8-4006-8aec-2fb6f4b1427b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:13.653833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:13.653833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:13.653833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:13.653833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:13.653833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:13.653833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:13.653833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:13.653833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:13.653833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:13.653833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:13.653833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:13.653833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:13.653833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:13.653833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:13.654790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:13.654790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:13.654790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:13.654790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:13.654790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:13.654790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:13.654790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:13.654790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:13.654790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:13.654790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:13.654790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:13.654790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:13.654790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:13.654790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:13.654790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:13.654790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:13.654790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:13.656134 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:13.656134 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:13.656134 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:13.656134 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:13.659614 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 429/858] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:14:13.661109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9be5e4-767e-492a-bac4-868c532d2b4d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:13.661362 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9be5e4-767e-492a-bac4-868c532d2b4d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:13.661635 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9be5e4-767e-492a-bac4-868c532d2b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:13.678099 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7e9be5e4-767e-492a-bac4-868c532d2b4d None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:13.679287 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7e9be5e4-767e-492a-bac4-868c532d2b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:13.680062 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:13.750490 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 430/859] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:13 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:13.751341 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9be5e4-767e-492a-bac4-868c532d2b4d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:13.752162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f91833-45ec-45bf-8b5e-753309c3b688 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:13.793014 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677d539-fe18-4d27-8e3e-dc1adfb27fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:13.793689 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677d539-fe18-4d27-8e3e-dc1adfb27fdb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:13.793995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677d539-fe18-4d27-8e3e-dc1adfb27fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:13.794276 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677d539-fe18-4d27-8e3e-dc1adfb27fdb None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:13.795006 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677d539-fe18-4d27-8e3e-dc1adfb27fdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:13.795316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677d539-fe18-4d27-8e3e-dc1adfb27fdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:13.797198 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677d539-fe18-4d27-8e3e-dc1adfb27fdb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:13.810485 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 430/860] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:13.811305 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677d539-fe18-4d27-8e3e-dc1adfb27fdb None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:13.839334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:13.857873 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9be5e4-767e-492a-bac4-868c532d2b4d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:13.891747 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c794926-1c45-42b7-ac30-dbc39e5ea70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:13.892407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c794926-1c45-42b7-ac30-dbc39e5ea70e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:13.892684 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c794926-1c45-42b7-ac30-dbc39e5ea70e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:13.892954 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c794926-1c45-42b7-ac30-dbc39e5ea70e None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:13.893662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c794926-1c45-42b7-ac30-dbc39e5ea70e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:13.893960 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c794926-1c45-42b7-ac30-dbc39e5ea70e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:13.896174 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c794926-1c45-42b7-ac30-dbc39e5ea70e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:13.907507 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 431/861] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:13.908831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a99cd1-b4d6-4156-972d-72a51e64ba55 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:13.909120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a99cd1-b4d6-4156-972d-72a51e64ba55 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:13.909224 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:13.909394 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a99cd1-b4d6-4156-972d-72a51e64ba55 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:13.910750 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 432/862] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:13.911348 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a99cd1-b4d6-4156-972d-72a51e64ba55 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:13.918241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677d539-fe18-4d27-8e3e-dc1adfb27fdb None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:13.951607 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5f8f4-d99d-4ee5-878a-729313059f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:13.952274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5f8f4-d99d-4ee5-878a-729313059f3e None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:13.952744 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5f8f4-d99d-4ee5-878a-729313059f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:13.953186 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5f8f4-d99d-4ee5-878a-729313059f3e None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:13.954228 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b5f8f4-d99d-4ee5-878a-729313059f3e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:13.955621 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b5f8f4-d99d-4ee5-878a-729313059f3e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:14.014563 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a99cd1-b4d6-4156-972d-72a51e64ba55 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:14.034015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 431/863] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 30 16:14:13 2022] POST /identity/v3/services => generated 251 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:14:14.034847 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5f8f4-d99d-4ee5-878a-729313059f3e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:14.059862 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae084d7-bd29-46a3-a81c-578aef786ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:14.060887 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae084d7-bd29-46a3-a81c-578aef786ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:14.061218 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae084d7-bd29-46a3-a81c-578aef786ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:14.061498 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae084d7-bd29-46a3-a81c-578aef786ce4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:14.065246 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:14.071250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae084d7-bd29-46a3-a81c-578aef786ce4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:14.073016 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae084d7-bd29-46a3-a81c-578aef786ce4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:14.088737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1ae084d7-bd29-46a3-a81c-578aef786ce4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:14.088737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:14.088737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:14.088737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:14.088737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:14.088737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:14.088737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:14.088737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:14.088737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:14.088737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:14.088737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:14.088737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:14.088737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:14:14.088737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:14:14.088737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:14:14.089757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:14:14.089757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:14.089757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:14.089757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:14.089757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:14.089757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:14.089757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:14.089757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:14.089757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:14.089757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:14.089757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:14.089757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:14.089757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:14.089757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:14.089757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:14.089757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:14.089757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:14.090788 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:14:14.090788 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:14:14.090788 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:14:14.090788 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:14:14.090788 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:14:14.090788 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:14:14.090788 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:14.090788 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:14.090788 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 433/864] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:14:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:14.091900 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d74f8b-6c5b-43d3-8a9f-b1c501618bdd None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:14.092192 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d74f8b-6c5b-43d3-8a9f-b1c501618bdd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:14.092472 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d74f8b-6c5b-43d3-8a9f-b1c501618bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:14.110652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-79d74f8b-6c5b-43d3-8a9f-b1c501618bdd None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:14.111839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-79d74f8b-6c5b-43d3-8a9f-b1c501618bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:14.172228 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5f8f4-d99d-4ee5-878a-729313059f3e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:14.188271 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 434/865] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:14 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:14.188754 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d74f8b-6c5b-43d3-8a9f-b1c501618bdd None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:14.210177 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35122a70-7dfe-403a-94ba-8050acf0bba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:14.211324 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35122a70-7dfe-403a-94ba-8050acf0bba9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:14.211895 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35122a70-7dfe-403a-94ba-8050acf0bba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:14.212378 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35122a70-7dfe-403a-94ba-8050acf0bba9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:14.221950 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35122a70-7dfe-403a-94ba-8050acf0bba9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:14.223671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35122a70-7dfe-403a-94ba-8050acf0bba9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:14.240122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-35122a70-7dfe-403a-94ba-8050acf0bba9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 16:14:14.240122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:14.240122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:14.240122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:14.240122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:14.240122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:14.240122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:14.240122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:14.240122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:14.240122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:14.240122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:14.240122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:14.240122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:14:14.240122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:14:14.240122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:14:14.240122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:14:14.241171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:14.241171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:14.241171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:14:14.241171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:14.241171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:14:14.241171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:14.241171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:14.241171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:14.241171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:14.241171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:14.241171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:14.241171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:14.241171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:14.241171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:14.241171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:14.241171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:14.241171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:14.241171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:14.242164 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:14.242164 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:14.242164 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:14:14.242164 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:14:14.242164 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:14:14.242164 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:14:14.242164 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:14:14.242164 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:14:14.242164 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 16:14:14.242164 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:14.242164 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 432/866] 10.222.0.67 () {56 vars in 1117 bytes} [Thu Jun 30 16:14:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:14.242777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35122a70-7dfe-403a-94ba-8050acf0bba9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:14.260248 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:14.311912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d74f8b-6c5b-43d3-8a9f-b1c501618bdd None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:14.342555 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:14.350801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35122a70-7dfe-403a-94ba-8050acf0bba9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:14.354319 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66927225-549d-458b-b924-c896b18dec17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:14.354319 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66927225-549d-458b-b924-c896b18dec17 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:14.354319 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66927225-549d-458b-b924-c896b18dec17 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:14.354319 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66927225-549d-458b-b924-c896b18dec17 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:14.354641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66927225-549d-458b-b924-c896b18dec17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:14.354641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66927225-549d-458b-b924-c896b18dec17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:14.355555 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66927225-549d-458b-b924-c896b18dec17 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:14.373133 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 435/867] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:14:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:14.373133 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66927225-549d-458b-b924-c896b18dec17 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:14.394794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b126e-c60e-40f6-9873-ed2577205147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:14.395821 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b126e-c60e-40f6-9873-ed2577205147 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:14.395996 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b126e-c60e-40f6-9873-ed2577205147 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:14.396279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b126e-c60e-40f6-9873-ed2577205147 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:14.397010 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4b126e-c60e-40f6-9873-ed2577205147 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:14.398745 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4b126e-c60e-40f6-9873-ed2577205147 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:14.475557 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66927225-549d-458b-b924-c896b18dec17 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:14.495502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 433/868] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 30 16:14:14 2022] POST /identity/v3/services => generated 269 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:14:14.495502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b126e-c60e-40f6-9873-ed2577205147 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:14.536999 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6e966-9932-46c8-8ee8-ed102c055568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:14.538191 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6e966-9932-46c8-8ee8-ed102c055568 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:14.543421 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6e966-9932-46c8-8ee8-ed102c055568 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:14.543421 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6e966-9932-46c8-8ee8-ed102c055568 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:14.543421 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6e966-9932-46c8-8ee8-ed102c055568 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:14.543421 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6e966-9932-46c8-8ee8-ed102c055568 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:14.543421 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6e966-9932-46c8-8ee8-ed102c055568 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:14.571086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 436/869] 10.222.0.67 () {56 vars in 1149 bytes} [Thu Jun 30 16:14:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:14.577568 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6e966-9932-46c8-8ee8-ed102c055568 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:14.596358 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:14.631990 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4b126e-c60e-40f6-9873-ed2577205147 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:14.638504 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:14.676842 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd4305-5484-4261-9c1a-1647f3b2b8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:14.678782 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd4305-5484-4261-9c1a-1647f3b2b8d4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:14.679295 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd4305-5484-4261-9c1a-1647f3b2b8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:14.679801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd4305-5484-4261-9c1a-1647f3b2b8d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:14.680856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd4305-5484-4261-9c1a-1647f3b2b8d4 None admin] RBAC: Adding query filter params (role.id=8acc34a163874a60b03728b814eaf075, scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d, user.id=988b917eea4b48ffae32e23b8555a5c6) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:14.681310 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd4305-5484-4261-9c1a-1647f3b2b8d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:14.683346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd4305-5484-4261-9c1a-1647f3b2b8d4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:14.692310 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 434/870] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:14:14 2022] GET /identity/v3/role_assignments?user.id=988b917eea4b48ffae32e23b8555a5c6&scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d&role.id=8acc34a163874a60b03728b814eaf075 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:14.729600 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6e966-9932-46c8-8ee8-ed102c055568 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:14.774621 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4753351-2fc4-43f0-a4a3-70e826d7110f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:14.775304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4753351-2fc4-43f0-a4a3-70e826d7110f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:14.775620 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4753351-2fc4-43f0-a4a3-70e826d7110f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:14.775899 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4753351-2fc4-43f0-a4a3-70e826d7110f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:14.856659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4753351-2fc4-43f0-a4a3-70e826d7110f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:14.858141 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4753351-2fc4-43f0-a4a3-70e826d7110f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:14.904596 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:14.971589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 437/871] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:14:14 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 16:14:14.974133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd4305-5484-4261-9c1a-1647f3b2b8d4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:15.091389 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd4305-5484-4261-9c1a-1647f3b2b8d4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:15.126931 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb06c46-b63f-47b6-bcd0-4bddf7df7b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:15.127773 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb06c46-b63f-47b6-bcd0-4bddf7df7b92 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:15.128054 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb06c46-b63f-47b6-bcd0-4bddf7df7b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:15.128327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb06c46-b63f-47b6-bcd0-4bddf7df7b92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:15.140146 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb06c46-b63f-47b6-bcd0-4bddf7df7b92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:15.141624 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb06c46-b63f-47b6-bcd0-4bddf7df7b92 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:15.149181 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:15.160923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-feb06c46-b63f-47b6-bcd0-4bddf7df7b92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:15.160923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:15.160923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:15.160923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:15.160923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:15.160923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:15.160923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:15.160923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:15.160923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:15.160923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:15.160923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:15.160923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:15.160923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:15.160923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:15.160923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:15.161799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:15.161799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:15.161799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:15.161799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:15.161799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:15.161799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:15.161799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:15.161799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:15.161799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:15.161799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:15.161799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:15.161799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:15.161799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:15.161799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:15.161799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:15.161799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:15.161799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:15.162887 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:15.162887 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:15.162887 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:15.162887 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:15.162887 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 435/872] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:14:15.169300 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4753351-2fc4-43f0-a4a3-70e826d7110f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:15.278573 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4753351-2fc4-43f0-a4a3-70e826d7110f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:15.314186 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e809e5c-3543-4dae-826e-236dcca14d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:15.314853 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e809e5c-3543-4dae-826e-236dcca14d7e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:15.315136 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e809e5c-3543-4dae-826e-236dcca14d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:15.315413 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e809e5c-3543-4dae-826e-236dcca14d7e None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:15.316157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e809e5c-3543-4dae-826e-236dcca14d7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:15.316465 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e809e5c-3543-4dae-826e-236dcca14d7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:15.318528 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e809e5c-3543-4dae-826e-236dcca14d7e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:15.327167 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:15.331295 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 438/873] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:15.340611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb06c46-b63f-47b6-bcd0-4bddf7df7b92 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:15.452881 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb06c46-b63f-47b6-bcd0-4bddf7df7b92 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:15.489219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90a3e3-5c0a-4891-8d99-4c92526024c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:15.492805 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90a3e3-5c0a-4891-8d99-4c92526024c0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:15.492805 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90a3e3-5c0a-4891-8d99-4c92526024c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:15.492805 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90a3e3-5c0a-4891-8d99-4c92526024c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:15.497660 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90a3e3-5c0a-4891-8d99-4c92526024c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:15.499116 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90a3e3-5c0a-4891-8d99-4c92526024c0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:15.514232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-7a90a3e3-5c0a-4891-8d99-4c92526024c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:15.514232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:15.514232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:15.514232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:15.514232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:15.514232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:15.514232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:15.514232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:15.514232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:15.514232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:15.514232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:15.514232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:15.514232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:14:15.514232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:14:15.514232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:14:15.515115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:14:15.515115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:15.515115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:15.515115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:15.515115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:15.515115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:15.515115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:15.515115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:15.515115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:15.515115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:15.515115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:15.515115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:15.515115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:15.515115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:15.515115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:15.515115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:15.515115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:15.516519 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:14:15.516519 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:14:15.516519 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:14:15.516519 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:14:15.516519 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:14:15.516519 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:14:15.516519 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:15.516519 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:15.516519 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 436/874] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:14:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:15.521463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e809e5c-3543-4dae-826e-236dcca14d7e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:15.624900 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e809e5c-3543-4dae-826e-236dcca14d7e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:15.659788 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067cd62a-582f-4751-b778-c1ad7a6f41fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:15.660547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067cd62a-582f-4751-b778-c1ad7a6f41fc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:15.660834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067cd62a-582f-4751-b778-c1ad7a6f41fc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:15.661112 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067cd62a-582f-4751-b778-c1ad7a6f41fc None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:15.661899 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067cd62a-582f-4751-b778-c1ad7a6f41fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:15.662211 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067cd62a-582f-4751-b778-c1ad7a6f41fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:15.663664 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067cd62a-582f-4751-b778-c1ad7a6f41fc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:15.677757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 439/875] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:14:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:15.683604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90a3e3-5c0a-4891-8d99-4c92526024c0 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:15.739683 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:15.793754 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90a3e3-5c0a-4891-8d99-4c92526024c0 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:15.828998 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051bfb84-299e-4698-8fdb-2fe3633f138a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:15.829658 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051bfb84-299e-4698-8fdb-2fe3633f138a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:15.829941 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051bfb84-299e-4698-8fdb-2fe3633f138a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:15.830216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051bfb84-299e-4698-8fdb-2fe3633f138a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:15.831043 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051bfb84-299e-4698-8fdb-2fe3633f138a None admin] RBAC: Adding query filter params (role.id=57bc5df632b3407d97116ae34d92a5cb, scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d, user.id=d49604bffa3d4e89a631eefa6d4e4f5d) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:15.831352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051bfb84-299e-4698-8fdb-2fe3633f138a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:15.833364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051bfb84-299e-4698-8fdb-2fe3633f138a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:15.839639 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 437/876] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:14:15 2022] GET /identity/v3/role_assignments?user.id=d49604bffa3d4e89a631eefa6d4e4f5d&scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d&role.id=57bc5df632b3407d97116ae34d92a5cb => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:16.227213 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63d234-5b7f-4e31-9f40-a071762215e5 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:16.227621 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63d234-5b7f-4e31-9f40-a071762215e5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:16.227827 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63d234-5b7f-4e31-9f40-a071762215e5 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:16.229493 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 440/877] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:16.234807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ef8f8-b62e-48ce-871e-7d632a57d51a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:16.235098 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ef8f8-b62e-48ce-871e-7d632a57d51a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:16.235377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ef8f8-b62e-48ce-871e-7d632a57d51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:16.252748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d99ef8f8-b62e-48ce-871e-7d632a57d51a None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:16.253851 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d99ef8f8-b62e-48ce-871e-7d632a57d51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:16.327147 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 438/878] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:16.350588 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf63d234-5b7f-4e31-9f40-a071762215e5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:16.420147 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:16.460433 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf63d234-5b7f-4e31-9f40-a071762215e5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:16.499932 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421987d6-2676-4879-96b7-41c49effb117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:16.500782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421987d6-2676-4879-96b7-41c49effb117 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:16.501065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421987d6-2676-4879-96b7-41c49effb117 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:16.501340 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421987d6-2676-4879-96b7-41c49effb117 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:16.501994 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421987d6-2676-4879-96b7-41c49effb117 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:16.503415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421987d6-2676-4879-96b7-41c49effb117 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:16.515847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-421987d6-2676-4879-96b7-41c49effb117 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 16:14:16.515847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:16.515847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:14:16.515847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:14:16.515847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:14:16.515847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:14:16.515847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:14:16.515847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:16.515847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 16:14:16.515847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:16.515847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:14:16.515847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:16.515847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:16.515847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:16.515847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:16.515847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:16.515847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:16.516906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:16.516906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:16.516906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:16.516906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:16.516906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:16.516906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:16.516906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:14:16.516906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:14:16.516906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:14:16.516906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:14:16.516906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:16.516906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:16.516906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:16.516906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:16.516906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:16.516906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:16.517990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:16.517990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:16.517990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:16.517990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:16.517990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:16.517990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:16.517990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:16.517990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:16.517990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:16.517990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:16.517990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:14:16.517990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:16.517990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 16:14:16.517990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:16.517990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 441/879] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 30 16:14:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:16.528806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ef8f8-b62e-48ce-871e-7d632a57d51a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:16.576835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:16.635000 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ef8f8-b62e-48ce-871e-7d632a57d51a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:16.672988 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac925ec-a035-4991-a24f-5a1666b42dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:16.673662 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac925ec-a035-4991-a24f-5a1666b42dff None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:16.673949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac925ec-a035-4991-a24f-5a1666b42dff None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:16.674223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac925ec-a035-4991-a24f-5a1666b42dff None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:16.674933 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac925ec-a035-4991-a24f-5a1666b42dff None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:16.675241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac925ec-a035-4991-a24f-5a1666b42dff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:16.676722 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac925ec-a035-4991-a24f-5a1666b42dff None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:16.682065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 439/880] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 30 16:14:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:16.687684 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421987d6-2676-4879-96b7-41c49effb117 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:16.742735 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d1d1b-0021-4289-b60f-45a6501c0eb7 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:16.743244 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d1d1b-0021-4289-b60f-45a6501c0eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:16.743319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d1d1b-0021-4289-b60f-45a6501c0eb7 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:16.744952 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 440/881] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:14:16.749826 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b6b42-f046-40e7-abec-bb3394683822 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:16.750119 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b6b42-f046-40e7-abec-bb3394683822 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:16.750403 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b6b42-f046-40e7-abec-bb3394683822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:16.773232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7b4b6b42-f046-40e7-abec-bb3394683822 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:16.774968 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7b4b6b42-f046-40e7-abec-bb3394683822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:16.798293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421987d6-2676-4879-96b7-41c49effb117 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:16.827809 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:16.845518 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc1c03-2b69-4130-b33c-9bf7d63bdf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:16.846201 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc1c03-2b69-4130-b33c-9bf7d63bdf60 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:16.846651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc1c03-2b69-4130-b33c-9bf7d63bdf60 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:16.846937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc1c03-2b69-4130-b33c-9bf7d63bdf60 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:16.848159 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc1c03-2b69-4130-b33c-9bf7d63bdf60 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:16.848797 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc1c03-2b69-4130-b33c-9bf7d63bdf60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:16.850874 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc1c03-2b69-4130-b33c-9bf7d63bdf60 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:16.857771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 442/882] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 30 16:14:16 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:16.863163 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc1c03-2b69-4130-b33c-9bf7d63bdf60 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:16.864930 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 441/883] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:16 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 16:14:16.886636 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b6b42-f046-40e7-abec-bb3394683822 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:16.951929 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:16.986488 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc1c03-2b69-4130-b33c-9bf7d63bdf60 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:17.020987 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b6b42-f046-40e7-abec-bb3394683822 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:17.026281 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d9011-65de-48d3-9891-d5cb33f4f4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:17.026879 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d9011-65de-48d3-9891-d5cb33f4f4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:17.027161 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d9011-65de-48d3-9891-d5cb33f4f4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:17.027433 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d9011-65de-48d3-9891-d5cb33f4f4a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:17.028250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d9011-65de-48d3-9891-d5cb33f4f4a4 None admin] RBAC: Adding query filter params (interface=public, service_id=185c0b2f524c4a2dacf445612098575d, region_id=RegionOne) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:17.028565 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d9011-65de-48d3-9891-d5cb33f4f4a4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:17.030452 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d9011-65de-48d3-9891-d5cb33f4f4a4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:17.038241 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 443/884] 10.222.0.67 () {56 vars in 1251 bytes} [Thu Jun 30 16:14:16 2022] GET /identity/v3/endpoints?service_id=185c0b2f524c4a2dacf445612098575d&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:17.039861 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f5462-66a3-4dee-89c4-ef7ba59e4ef6 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:17.040427 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f5462-66a3-4dee-89c4-ef7ba59e4ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:17.040926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f5462-66a3-4dee-89c4-ef7ba59e4ef6 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:17.042683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 444/885] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:17.044553 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f5462-66a3-4dee-89c4-ef7ba59e4ef6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:17.061977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0697be65-1981-4e04-9cc3-6f9b28a2a5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:17.062856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697be65-1981-4e04-9cc3-6f9b28a2a5db None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:17.063142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697be65-1981-4e04-9cc3-6f9b28a2a5db None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:17.063438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697be65-1981-4e04-9cc3-6f9b28a2a5db None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:17.064112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0697be65-1981-4e04-9cc3-6f9b28a2a5db None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:17.065615 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0697be65-1981-4e04-9cc3-6f9b28a2a5db None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:17.078559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-0697be65-1981-4e04-9cc3-6f9b28a2a5db None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 16:14:17.078559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:17.078559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:14:17.078559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:14:17.078559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:14:17.078559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:14:17.078559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:14:17.078559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:17.078559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 16:14:17.078559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:17.078559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:14:17.078559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:17.078559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:17.078559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:17.078559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:17.078559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:17.079759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:17.079759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:17.079759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:17.079759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:17.079759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:17.079759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:17.079759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:17.079759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:14:17.079759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:14:17.079759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:14:17.079759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:14:17.079759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:17.079759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:17.079759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:17.079759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:17.079759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:17.080667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:17.080667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:17.080667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:17.080667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:17.080667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:17.080667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:17.080667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:17.080667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:17.080667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:17.080667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:17.080667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:17.080667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:14:17.080667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:17.080667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 16:14:17.080667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:17.080667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 442/886] 10.222.0.67 () {56 vars in 1132 bytes} [Thu Jun 30 16:14:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:17.081505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cce8d4-fc27-4c9f-9925-74240fcef8b7 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:17.081505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cce8d4-fc27-4c9f-9925-74240fcef8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:17.081783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cce8d4-fc27-4c9f-9925-74240fcef8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:17.101991 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-63cce8d4-fc27-4c9f-9925-74240fcef8b7 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:17.103233 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-63cce8d4-fc27-4c9f-9925-74240fcef8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:17.112948 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:17.165922 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f5462-66a3-4dee-89c4-ef7ba59e4ef6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:17.178257 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 443/887] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:17 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:17.179057 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cce8d4-fc27-4c9f-9925-74240fcef8b7 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:17.207978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2bbe51-6fe6-4c6f-a516-fef9af066f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:17.209081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2bbe51-6fe6-4c6f-a516-fef9af066f98 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:17.209581 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2bbe51-6fe6-4c6f-a516-fef9af066f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:17.210147 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2bbe51-6fe6-4c6f-a516-fef9af066f98 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:17.210782 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:17.211973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2bbe51-6fe6-4c6f-a516-fef9af066f98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:17.213548 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2bbe51-6fe6-4c6f-a516-fef9af066f98 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:17.220632 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 445/888] 10.222.0.67 () {56 vars in 1087 bytes} [Thu Jun 30 16:14:17 2022] GET /identity/v3/services => generated 1268 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:14:17.224831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2bbe51-6fe6-4c6f-a516-fef9af066f98 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:17.297645 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cce8d4-fc27-4c9f-9925-74240fcef8b7 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:17.341700 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96cbade-fe09-4b81-a7f3-9eb1b47d0786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:17.343397 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cbade-fe09-4b81-a7f3-9eb1b47d0786 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:17.343876 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cbade-fe09-4b81-a7f3-9eb1b47d0786 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:17.344326 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cbade-fe09-4b81-a7f3-9eb1b47d0786 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:17.345717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96cbade-fe09-4b81-a7f3-9eb1b47d0786 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:17.346270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96cbade-fe09-4b81-a7f3-9eb1b47d0786 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:17.349044 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96cbade-fe09-4b81-a7f3-9eb1b47d0786 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:17.361878 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 444/889] 10.222.0.67 () {56 vars in 1126 bytes} [Thu Jun 30 16:14:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:17.362575 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96cbade-fe09-4b81-a7f3-9eb1b47d0786 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:17.379881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2bbe51-6fe6-4c6f-a516-fef9af066f98 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:17.401343 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:17.424762 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5793618-2ff7-43ae-bec8-63b9e991ed6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:17.425581 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5793618-2ff7-43ae-bec8-63b9e991ed6e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:17.425910 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5793618-2ff7-43ae-bec8-63b9e991ed6e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:17.426186 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5793618-2ff7-43ae-bec8-63b9e991ed6e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:17.427065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5793618-2ff7-43ae-bec8-63b9e991ed6e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:17.428473 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5793618-2ff7-43ae-bec8-63b9e991ed6e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:17.444469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a5793618-2ff7-43ae-bec8-63b9e991ed6e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 16:14:17.444469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:17.444469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:14:17.444469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:14:17.444469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:14:17.444469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:14:17.444469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:14:17.444469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:17.444469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 16:14:17.444469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:17.444469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:14:17.444469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:17.444469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:17.444469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:17.444469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:17.444469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:17.444469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:17.445497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:17.445497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:17.445497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:17.445497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:17.445497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:17.445497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:17.445497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:14:17.445497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:14:17.445497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:14:17.445497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:14:17.445497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:17.445497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:17.445497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:17.445497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:17.445497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:17.445497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:17.446334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:17.446334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:17.446334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:17.446334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:17.446334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:17.446334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:17.446334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:17.446334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:17.446334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:17.446334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:17.446334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:14:17.446334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:17.446334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 16:14:17.446334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:17.446334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 446/890] 10.222.0.67 () {56 vars in 1126 bytes} [Thu Jun 30 16:14:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:17.452065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5793618-2ff7-43ae-bec8-63b9e991ed6e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:17.487413 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:17.494099 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96cbade-fe09-4b81-a7f3-9eb1b47d0786 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:17.534157 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2085b2-3724-4e04-83b3-845b003d8ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:17.535179 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2085b2-3724-4e04-83b3-845b003d8ebb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:17.535642 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2085b2-3724-4e04-83b3-845b003d8ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:17.536024 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2085b2-3724-4e04-83b3-845b003d8ebb None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:17.536849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2085b2-3724-4e04-83b3-845b003d8ebb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:17.537353 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2085b2-3724-4e04-83b3-845b003d8ebb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:17.538940 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2085b2-3724-4e04-83b3-845b003d8ebb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:17.544997 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 445/891] 10.222.0.67 () {56 vars in 1126 bytes} [Thu Jun 30 16:14:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:17.551549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2085b2-3724-4e04-83b3-845b003d8ebb None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:17.586782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5793618-2ff7-43ae-bec8-63b9e991ed6e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:17.625023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c037cb-3dd3-4d86-9145-801a47cc7e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:17.625806 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c037cb-3dd3-4d86-9145-801a47cc7e79 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:17.626091 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c037cb-3dd3-4d86-9145-801a47cc7e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:17.626360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c037cb-3dd3-4d86-9145-801a47cc7e79 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:17.627167 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c037cb-3dd3-4d86-9145-801a47cc7e79 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:17.627488 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c037cb-3dd3-4d86-9145-801a47cc7e79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:17.629542 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c037cb-3dd3-4d86-9145-801a47cc7e79 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:17.635446 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 447/892] 10.222.0.67 () {56 vars in 1122 bytes} [Thu Jun 30 16:14:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:17.641252 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c037cb-3dd3-4d86-9145-801a47cc7e79 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:17.688520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2085b2-3724-4e04-83b3-845b003d8ebb None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:17.689999 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:17.728847 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eaa0be-da13-4942-96f5-0118e3c7822f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:17.729853 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eaa0be-da13-4942-96f5-0118e3c7822f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:17.729990 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eaa0be-da13-4942-96f5-0118e3c7822f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:17.730230 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eaa0be-da13-4942-96f5-0118e3c7822f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:17.731193 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eaa0be-da13-4942-96f5-0118e3c7822f None admin] RBAC: Adding query filter params (interface=public, service_id=e6244dd30bb749ebb60fd3c7ffed8a89, region_id=RegionOne) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:17.731453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eaa0be-da13-4942-96f5-0118e3c7822f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:17.732886 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eaa0be-da13-4942-96f5-0118e3c7822f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:17.743288 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 446/893] 10.222.0.67 () {56 vars in 1251 bytes} [Thu Jun 30 16:14:17 2022] GET /identity/v3/endpoints?service_id=e6244dd30bb749ebb60fd3c7ffed8a89&interface=public®ion_id=RegionOne => generated 194 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:17.747890 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eaa0be-da13-4942-96f5-0118e3c7822f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:17.756546 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c037cb-3dd3-4d86-9145-801a47cc7e79 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:17.787133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:17.794583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add014dc-8320-4771-a639-6417e60a6170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:17.795278 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add014dc-8320-4771-a639-6417e60a6170 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:17.795593 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add014dc-8320-4771-a639-6417e60a6170 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:17.796121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add014dc-8320-4771-a639-6417e60a6170 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:17.796643 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add014dc-8320-4771-a639-6417e60a6170 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:17.796997 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add014dc-8320-4771-a639-6417e60a6170 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:17.801965 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add014dc-8320-4771-a639-6417e60a6170 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:17.805013 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 448/894] 10.222.0.67 () {56 vars in 1122 bytes} [Thu Jun 30 16:14:17 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:17.884103 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eaa0be-da13-4942-96f5-0118e3c7822f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:17.926139 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5540f948-e39d-4ff0-8c12-f71a6774618a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:17.927159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540f948-e39d-4ff0-8c12-f71a6774618a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:17.927662 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540f948-e39d-4ff0-8c12-f71a6774618a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:17.928090 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540f948-e39d-4ff0-8c12-f71a6774618a None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:17.929209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540f948-e39d-4ff0-8c12-f71a6774618a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:17.930799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5540f948-e39d-4ff0-8c12-f71a6774618a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:17.938399 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 447/895] 10.222.0.67 () {56 vars in 1087 bytes} [Thu Jun 30 16:14:17 2022] GET /identity/v3/services => generated 1268 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:14:18.078951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72e868-cf28-408a-a9ef-03c70d014be7 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:18.079242 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72e868-cf28-408a-a9ef-03c70d014be7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:18.079558 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72e868-cf28-408a-a9ef-03c70d014be7 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:18.081260 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 449/896] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:18.089231 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950021f7-75c6-4f80-b24f-111b8a8640a9 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:18.089741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950021f7-75c6-4f80-b24f-111b8a8640a9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:18.090127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950021f7-75c6-4f80-b24f-111b8a8640a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:18.108732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-950021f7-75c6-4f80-b24f-111b8a8640a9 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:18.109935 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-950021f7-75c6-4f80-b24f-111b8a8640a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:18.185641 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 448/897] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:18 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:18.210881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72e868-cf28-408a-a9ef-03c70d014be7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:18.321223 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72e868-cf28-408a-a9ef-03c70d014be7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:18.330608 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:18.357831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915bbee-999c-45dc-9103-138f47d3da57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:18.358703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915bbee-999c-45dc-9103-138f47d3da57 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:18.358980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915bbee-999c-45dc-9103-138f47d3da57 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:18.359340 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1915bbee-999c-45dc-9103-138f47d3da57 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:18.370802 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1915bbee-999c-45dc-9103-138f47d3da57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:14:18.379665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1915bbee-999c-45dc-9103-138f47d3da57 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 16:14:18.379665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:18.379665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:18.379665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:18.379665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:18.379665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:18.379665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:18.379665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:18.379665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:18.379665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:18.379665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:18.379665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:18.379665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:14:18.379665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:14:18.379665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:14:18.379665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:14:18.380792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:14:18.380792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:14:18.380792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:14:18.380792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:14:18.380792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:18.380792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:18.380792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:18.380792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:18.380792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:18.380792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:18.380792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:18.380792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:18.380792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:18.380792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:18.380792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:18.380792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:18.381639 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:18.381639 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:18.381639 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:18.381639 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:18.381639 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:14:18.381639 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:14:18.381639 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:14:18.381639 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:14:18.381639 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:14:18.381639 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:14:18.381639 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 16:14:18.381639 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:18.381639 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 450/898] 10.222.0.67 () {56 vars in 1120 bytes} [Thu Jun 30 16:14:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:18.388820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950021f7-75c6-4f80-b24f-111b8a8640a9 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:18.409889 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:18.503201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950021f7-75c6-4f80-b24f-111b8a8640a9 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:18.540467 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36333813-0f92-4e64-8bf3-e5c669b0bb51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:18.541200 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36333813-0f92-4e64-8bf3-e5c669b0bb51 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:18.541482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36333813-0f92-4e64-8bf3-e5c669b0bb51 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:18.541806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36333813-0f92-4e64-8bf3-e5c669b0bb51 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:18.542559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36333813-0f92-4e64-8bf3-e5c669b0bb51 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:18.542870 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36333813-0f92-4e64-8bf3-e5c669b0bb51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:18.544294 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36333813-0f92-4e64-8bf3-e5c669b0bb51 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:18.554514 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 449/899] 10.222.0.67 () {56 vars in 1115 bytes} [Thu Jun 30 16:14:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:18.560081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915bbee-999c-45dc-9103-138f47d3da57 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:18.670187 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1915bbee-999c-45dc-9103-138f47d3da57 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:18.705065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62104fe2-ce60-41a3-bc1d-02fb40c7be7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:18.705854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62104fe2-ce60-41a3-bc1d-02fb40c7be7b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:18.706137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62104fe2-ce60-41a3-bc1d-02fb40c7be7b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:18.706411 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62104fe2-ce60-41a3-bc1d-02fb40c7be7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:18.717827 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62104fe2-ce60-41a3-bc1d-02fb40c7be7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:18.719280 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62104fe2-ce60-41a3-bc1d-02fb40c7be7b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:18.738073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-62104fe2-ce60-41a3-bc1d-02fb40c7be7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:18.738073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:18.738073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:18.738073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:18.738073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:18.738073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:18.738073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:18.738073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:18.738073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:18.738073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:18.738073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:18.738073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:18.738073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:18.738073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:18.738073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:18.738998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:18.738998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:18.738998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:18.738998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:18.738998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:18.738998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:18.738998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:18.738998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:18.738998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:18.738998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:18.738998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:18.738998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:18.738998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:18.738998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:18.738998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:18.738998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:18.738998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:18.740394 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:18.740394 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:18.740394 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:18.740394 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:18.740394 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 451/900] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:18.746434 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36333813-0f92-4e64-8bf3-e5c669b0bb51 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:18.832505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:18.852889 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36333813-0f92-4e64-8bf3-e5c669b0bb51 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:18.890477 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af92107-4d09-4aa3-94b6-7b9f27f5e4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:18.891149 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af92107-4d09-4aa3-94b6-7b9f27f5e4ad None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:18.891492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af92107-4d09-4aa3-94b6-7b9f27f5e4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:18.891779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af92107-4d09-4aa3-94b6-7b9f27f5e4ad None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:18.892502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af92107-4d09-4aa3-94b6-7b9f27f5e4ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:18.892809 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af92107-4d09-4aa3-94b6-7b9f27f5e4ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:18.894187 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af92107-4d09-4aa3-94b6-7b9f27f5e4ad None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:18.907116 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 450/901] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:18.912943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62104fe2-ce60-41a3-bc1d-02fb40c7be7b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:18.937532 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:19.029770 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62104fe2-ce60-41a3-bc1d-02fb40c7be7b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:19.064854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ede1d-d4b8-45cb-97e2-7cb0843c779d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:19.065668 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ede1d-d4b8-45cb-97e2-7cb0843c779d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:19.065949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ede1d-d4b8-45cb-97e2-7cb0843c779d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:19.066223 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ede1d-d4b8-45cb-97e2-7cb0843c779d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:19.074514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ede1d-d4b8-45cb-97e2-7cb0843c779d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:19.075996 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ede1d-d4b8-45cb-97e2-7cb0843c779d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:19.093732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-3e4ede1d-d4b8-45cb-97e2-7cb0843c779d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 16:14:19.093732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:19.093732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:19.093732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:19.093732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:19.093732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:19.093732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:19.093732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:19.093732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:19.093732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:19.093732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:19.093732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:19.093732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:14:19.093732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:14:19.093732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:14:19.093732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:14:19.094732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:19.094732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:19.094732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:14:19.094732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:19.094732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:14:19.094732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:19.094732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:19.094732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:19.094732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:19.094732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:19.094732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:19.094732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:19.094732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:19.094732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:19.094732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:19.094732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:19.094732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:19.094732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:19.096123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:19.096123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:19.096123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:14:19.096123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:14:19.096123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:14:19.096123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:14:19.096123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:14:19.096123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:14:19.096123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 16:14:19.096123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:19.096123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 452/902] 10.222.0.67 () {56 vars in 1117 bytes} [Thu Jun 30 16:14:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:19.101227 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af92107-4d09-4aa3-94b6-7b9f27f5e4ad None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:19.210178 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af92107-4d09-4aa3-94b6-7b9f27f5e4ad None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:19.247269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b63ac0-16b3-4953-8842-a694063fe31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:19.248029 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b63ac0-16b3-4953-8842-a694063fe31c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:19.248320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b63ac0-16b3-4953-8842-a694063fe31c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:19.248600 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b63ac0-16b3-4953-8842-a694063fe31c None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:19.249491 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b63ac0-16b3-4953-8842-a694063fe31c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:19.249807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b63ac0-16b3-4953-8842-a694063fe31c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:19.251250 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b63ac0-16b3-4953-8842-a694063fe31c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:19.262426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:19.274292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 451/903] 10.222.0.67 () {56 vars in 1149 bytes} [Thu Jun 30 16:14:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:19.280053 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ede1d-d4b8-45cb-97e2-7cb0843c779d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:19.370840 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:19.390746 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ede1d-d4b8-45cb-97e2-7cb0843c779d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:19.427728 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333d966-065f-4386-b6d1-b7e7b5a21b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:19.428378 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333d966-065f-4386-b6d1-b7e7b5a21b83 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:19.428660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333d966-065f-4386-b6d1-b7e7b5a21b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:19.428938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333d966-065f-4386-b6d1-b7e7b5a21b83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:19.499572 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333d966-065f-4386-b6d1-b7e7b5a21b83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:19.501038 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333d966-065f-4386-b6d1-b7e7b5a21b83 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:19.516996 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffebe4-7449-49cd-8078-a49710d7cf8f None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:19.517309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffebe4-7449-49cd-8078-a49710d7cf8f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:19.517571 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbffebe4-7449-49cd-8078-a49710d7cf8f None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:19.519062 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 452/904] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:19.525204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ef473-fa49-4499-a3d4-e146b68c61bc None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:19.525488 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ef473-fa49-4499-a3d4-e146b68c61bc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:19.525881 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ef473-fa49-4499-a3d4-e146b68c61bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:19.549790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e43ef473-fa49-4499-a3d4-e146b68c61bc None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:19.552385 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e43ef473-fa49-4499-a3d4-e146b68c61bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:19.593815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:19.619019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 453/905] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:14:19 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 16:14:19.620593 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333d966-065f-4386-b6d1-b7e7b5a21b83 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:19.658276 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 453/906] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:19 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 16:14:19.675566 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ef473-fa49-4499-a3d4-e146b68c61bc None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:19.750241 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333d966-065f-4386-b6d1-b7e7b5a21b83 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:19.786620 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ef473-fa49-4499-a3d4-e146b68c61bc None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:19.791392 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949b01e-834b-42ca-acf3-ada83cd34f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:19.792493 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949b01e-834b-42ca-acf3-ada83cd34f8e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:19.792999 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949b01e-834b-42ca-acf3-ada83cd34f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:19.793480 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949b01e-834b-42ca-acf3-ada83cd34f8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:19.805482 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949b01e-834b-42ca-acf3-ada83cd34f8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:19.807444 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949b01e-834b-42ca-acf3-ada83cd34f8e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:19.817511 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:19.825756 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e076e1c-731e-4306-8b24-c9b8f4ab4646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:19.826660 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e076e1c-731e-4306-8b24-c9b8f4ab4646 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:19.853853 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e076e1c-731e-4306-8b24-c9b8f4ab4646 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:19.853853 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e076e1c-731e-4306-8b24-c9b8f4ab4646 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:19.853853 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e076e1c-731e-4306-8b24-c9b8f4ab4646 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:19.853853 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e076e1c-731e-4306-8b24-c9b8f4ab4646 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:19.854172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-b949b01e-834b-42ca-acf3-ada83cd34f8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:19.854172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:19.854172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:19.854172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:19.854172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:19.854172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:19.854172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:19.854172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:19.854172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:19.854172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:19.854172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:19.854172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:19.854172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:19.854172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:19.854172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:19.855045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:19.855045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:19.855045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:19.855045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:19.855045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:19.855045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:19.855045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:19.855045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:19.855045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:19.855045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:19.855045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:19.855045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:19.855045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:19.855045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:19.855045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:19.855045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:19.855045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:19.855948 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:19.855948 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:19.855948 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:19.855948 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:19.855948 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 454/907] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:19.855948 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949b01e-834b-42ca-acf3-ada83cd34f8e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:19.856275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-7e076e1c-731e-4306-8b24-c9b8f4ab4646 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 16:14:19.856275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:19.856275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:14:19.856275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:14:19.856275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:14:19.856275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:14:19.856275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:14:19.856275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:19.856275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 16:14:19.856275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:19.856275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:14:19.856275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:19.856275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:19.856275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:19.856275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:19.856275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:19.856275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:19.857156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:19.857156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:19.857156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:19.857156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:19.857156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:19.857156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:19.857156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:14:19.857156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:14:19.857156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:14:19.857156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:14:19.857156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:19.857156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:19.857156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:19.857156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:19.857156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:19.857156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:19.857970 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:19.857970 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:19.857970 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:19.857970 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:19.857970 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:19.857970 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:19.857970 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:19.857970 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:19.857970 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:19.857970 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:19.857970 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:14:19.857970 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:19.857970 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 16:14:19.857970 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:19.857970 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 454/908] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 30 16:14:19 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:19.858768 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e076e1c-731e-4306-8b24-c9b8f4ab4646 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:19.889756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:19.969207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949b01e-834b-42ca-acf3-ada83cd34f8e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:19.993548 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e076e1c-731e-4306-8b24-c9b8f4ab4646 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:20.015005 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c99f17-d766-4f05-a2eb-1495afbe75f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:20.015632 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c99f17-d766-4f05-a2eb-1495afbe75f2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:20.015921 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c99f17-d766-4f05-a2eb-1495afbe75f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:20.016203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c99f17-d766-4f05-a2eb-1495afbe75f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:20.016894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c99f17-d766-4f05-a2eb-1495afbe75f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:20.017201 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c99f17-d766-4f05-a2eb-1495afbe75f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:20.018568 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c99f17-d766-4f05-a2eb-1495afbe75f2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:20.031297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 455/909] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:20.032829 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b03f0-704e-44ec-93cb-75edb0edc3c5 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:20.033118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b03f0-704e-44ec-93cb-75edb0edc3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:20.033392 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b03f0-704e-44ec-93cb-75edb0edc3c5 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:20.034761 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 456/910] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:20.039704 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5fc3b-99c5-4659-a86e-72c50c5d5cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:20.039906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b03f0-704e-44ec-93cb-75edb0edc3c5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:20.040454 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5fc3b-99c5-4659-a86e-72c50c5d5cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:20.040738 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5fc3b-99c5-4659-a86e-72c50c5d5cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:20.041014 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5fc3b-99c5-4659-a86e-72c50c5d5cf7 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:20.041730 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b5fc3b-99c5-4659-a86e-72c50c5d5cf7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:20.042039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b5fc3b-99c5-4659-a86e-72c50c5d5cf7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:20.044688 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b5fc3b-99c5-4659-a86e-72c50c5d5cf7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:20.055423 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 455/911] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 30 16:14:19 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:20.055423 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01928bf0-e58d-4b62-bf16-4997d7d67464 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:20.055423 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01928bf0-e58d-4b62-bf16-4997d7d67464 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:20.055423 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01928bf0-e58d-4b62-bf16-4997d7d67464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:20.072311 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-01928bf0-e58d-4b62-bf16-4997d7d67464 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:20.073498 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-01928bf0-e58d-4b62-bf16-4997d7d67464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:20.134000 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:20.141729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:20.153857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b03f0-704e-44ec-93cb-75edb0edc3c5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:20.163493 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 456/912] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:20 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:20.163493 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01928bf0-e58d-4b62-bf16-4997d7d67464 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:20.199805 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53de87-2ae1-45f1-9387-5caf116f70d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:20.200595 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53de87-2ae1-45f1-9387-5caf116f70d6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:20.200882 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53de87-2ae1-45f1-9387-5caf116f70d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:20.201153 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53de87-2ae1-45f1-9387-5caf116f70d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:20.210256 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53de87-2ae1-45f1-9387-5caf116f70d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:20.211729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53de87-2ae1-45f1-9387-5caf116f70d6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:20.226935 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-4c53de87-2ae1-45f1-9387-5caf116f70d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:20.226935 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:20.226935 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:20.226935 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:20.226935 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:20.226935 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:20.226935 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:20.226935 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:20.226935 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:20.226935 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:20.226935 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:20.226935 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:20.226935 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:14:20.226935 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:14:20.226935 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:14:20.228040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:14:20.228040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:20.228040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:20.228040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:20.228040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:20.228040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:20.228040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:20.228040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:20.228040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:20.228040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:20.228040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:20.228040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:20.228040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:20.228040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:20.228040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:20.228040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:20.228040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:20.228917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:14:20.228917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:14:20.228917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:14:20.228917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:14:20.228917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:14:20.228917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:14:20.228917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:20.228917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:20.228917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 457/913] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:14:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:20.229394 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53de87-2ae1-45f1-9387-5caf116f70d6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:20.274713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01928bf0-e58d-4b62-bf16-4997d7d67464 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:20.309183 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c605b14-0c2f-44c0-bd9b-3793719712e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:20.309838 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c605b14-0c2f-44c0-bd9b-3793719712e3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:20.310121 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c605b14-0c2f-44c0-bd9b-3793719712e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:20.310396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c605b14-0c2f-44c0-bd9b-3793719712e3 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:20.311096 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c605b14-0c2f-44c0-bd9b-3793719712e3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:20.311399 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c605b14-0c2f-44c0-bd9b-3793719712e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:20.312798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c605b14-0c2f-44c0-bd9b-3793719712e3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:20.317952 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 457/914] 10.222.0.67 () {56 vars in 1114 bytes} [Thu Jun 30 16:14:20 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:20.319079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf189c1-deba-4805-9440-196616d2abdc None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:20.319366 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf189c1-deba-4805-9440-196616d2abdc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:20.319712 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf189c1-deba-4805-9440-196616d2abdc None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:20.322739 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 458/915] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:20.322739 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf189c1-deba-4805-9440-196616d2abdc None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:20.332990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53de87-2ae1-45f1-9387-5caf116f70d6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:20.367054 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d199d-5005-468e-8227-0e24ae4a2b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:20.367748 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d199d-5005-468e-8227-0e24ae4a2b2d None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:20.368031 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d199d-5005-468e-8227-0e24ae4a2b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:20.368306 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d199d-5005-468e-8227-0e24ae4a2b2d None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:20.369086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d199d-5005-468e-8227-0e24ae4a2b2d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:20.370524 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d199d-5005-468e-8227-0e24ae4a2b2d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:20.376602 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:20.429089 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf189c1-deba-4805-9440-196616d2abdc None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:20.446371 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 458/916] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 30 16:14:20 2022] POST /identity/v3/services => generated 241 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:14:20.447180 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d199d-5005-468e-8227-0e24ae4a2b2d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:20.465396 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:20.473473 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316c093-b188-4ca9-b675-7e5d824f513f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:20.474253 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316c093-b188-4ca9-b675-7e5d824f513f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:20.474660 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316c093-b188-4ca9-b675-7e5d824f513f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:20.474891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316c093-b188-4ca9-b675-7e5d824f513f None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:20.475904 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316c093-b188-4ca9-b675-7e5d824f513f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:20.476222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316c093-b188-4ca9-b675-7e5d824f513f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:20.477967 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316c093-b188-4ca9-b675-7e5d824f513f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:20.492809 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 459/917] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:14:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:20.494394 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76438e7-995e-4671-8ea0-8a2a564031af None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:20.494830 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76438e7-995e-4671-8ea0-8a2a564031af None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:20.495202 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76438e7-995e-4671-8ea0-8a2a564031af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:20.515205 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c76438e7-995e-4671-8ea0-8a2a564031af None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:20.516626 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c76438e7-995e-4671-8ea0-8a2a564031af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:20.571898 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d199d-5005-468e-8227-0e24ae4a2b2d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:20.599998 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 460/918] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:20 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:20.600881 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76438e7-995e-4671-8ea0-8a2a564031af None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:20.613489 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64b909-e486-4ec3-915f-b837912483e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:20.614388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64b909-e486-4ec3-915f-b837912483e8 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:20.614678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64b909-e486-4ec3-915f-b837912483e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:20.614959 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b64b909-e486-4ec3-915f-b837912483e8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:20.615738 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64b909-e486-4ec3-915f-b837912483e8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:20.617123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b64b909-e486-4ec3-915f-b837912483e8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:20.642722 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:20.698658 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 459/919] 10.222.0.67 () {60 vars in 1144 bytes} [Thu Jun 30 16:14:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:14:20.699890 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64b909-e486-4ec3-915f-b837912483e8 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:20.714637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76438e7-995e-4671-8ea0-8a2a564031af None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:20.765222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93a54e-83b2-422c-b466-60f2599d0cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:20.766120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93a54e-83b2-422c-b466-60f2599d0cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:20.766510 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93a54e-83b2-422c-b466-60f2599d0cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:20.766889 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c93a54e-83b2-422c-b466-60f2599d0cf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:20.767916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93a54e-83b2-422c-b466-60f2599d0cf8 None admin] RBAC: Adding query filter params (role.id=c2206d1398e84c28b5365a335a04e5ac, scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d, user.id=d49604bffa3d4e89a631eefa6d4e4f5d) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:20.768343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93a54e-83b2-422c-b466-60f2599d0cf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:20.770688 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c93a54e-83b2-422c-b466-60f2599d0cf8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:20.783911 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 461/920] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:14:20 2022] GET /identity/v3/role_assignments?user.id=d49604bffa3d4e89a631eefa6d4e4f5d&scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d&role.id=c2206d1398e84c28b5365a335a04e5ac => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:20.805458 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:20.856345 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b64b909-e486-4ec3-915f-b837912483e8 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:20.901470 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d57664-78ca-4609-a1d5-8bd8ee0480db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:20.902351 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d57664-78ca-4609-a1d5-8bd8ee0480db None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:20.902631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d57664-78ca-4609-a1d5-8bd8ee0480db None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:20.902899 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d57664-78ca-4609-a1d5-8bd8ee0480db None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:20.903598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d57664-78ca-4609-a1d5-8bd8ee0480db None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:20.905011 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d57664-78ca-4609-a1d5-8bd8ee0480db None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:20.919359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-c7d57664-78ca-4609-a1d5-8bd8ee0480db None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 16:14:20.919359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:20.919359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:14:20.919359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:14:20.919359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:14:20.919359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:14:20.919359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:14:20.919359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:20.919359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 16:14:20.919359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:20.919359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:14:20.919359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:20.919359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:20.919359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:20.919359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:20.919359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:20.920414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:20.920414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:20.920414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:20.920414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:20.920414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:20.920414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:20.920414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:20.920414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:14:20.920414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:14:20.920414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:14:20.920414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:14:20.920414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:20.920414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:20.920414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:20.920414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:20.920414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:20.921503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:20.921503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:20.921503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:20.921503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:20.921503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:20.921503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:20.921503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:20.921503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:20.921503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:20.921503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:20.921503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:20.921503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:14:20.921503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:20.921503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 16:14:20.921503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:20.921503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 460/921] 10.222.0.67 () {56 vars in 1132 bytes} [Thu Jun 30 16:14:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:20.927517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93a54e-83b2-422c-b466-60f2599d0cf8 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:21.039907 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:21.045626 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c93a54e-83b2-422c-b466-60f2599d0cf8 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:21.082679 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d91d53-5344-4275-9434-571a29138cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:21.083730 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d91d53-5344-4275-9434-571a29138cca None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:21.084430 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d91d53-5344-4275-9434-571a29138cca None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:21.084976 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d91d53-5344-4275-9434-571a29138cca None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:21.086012 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d91d53-5344-4275-9434-571a29138cca None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:21.086583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d91d53-5344-4275-9434-571a29138cca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:21.088800 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d91d53-5344-4275-9434-571a29138cca None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:21.094334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 462/922] 10.222.0.67 () {56 vars in 1126 bytes} [Thu Jun 30 16:14:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:21.100435 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d57664-78ca-4609-a1d5-8bd8ee0480db None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:21.215007 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d57664-78ca-4609-a1d5-8bd8ee0480db None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:21.247001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:21.252281 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059d7b96-84ab-43a3-ad22-e55af608029b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:21.252946 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d7b96-84ab-43a3-ad22-e55af608029b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:21.253229 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d7b96-84ab-43a3-ad22-e55af608029b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:21.253503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059d7b96-84ab-43a3-ad22-e55af608029b None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:21.257318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d7b96-84ab-43a3-ad22-e55af608029b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:21.257318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d7b96-84ab-43a3-ad22-e55af608029b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:21.257318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059d7b96-84ab-43a3-ad22-e55af608029b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:21.262625 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 461/923] 10.222.0.67 () {56 vars in 1126 bytes} [Thu Jun 30 16:14:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:21.268572 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d91d53-5344-4275-9434-571a29138cca None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:21.383303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d91d53-5344-4275-9434-571a29138cca None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:21.419714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eabc9d9-ad55-41ba-a5ae-20ee35a63b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:21.420380 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabc9d9-ad55-41ba-a5ae-20ee35a63b06 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:21.420666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabc9d9-ad55-41ba-a5ae-20ee35a63b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:21.420941 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabc9d9-ad55-41ba-a5ae-20ee35a63b06 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:21.421688 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eabc9d9-ad55-41ba-a5ae-20ee35a63b06 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:21.423071 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eabc9d9-ad55-41ba-a5ae-20ee35a63b06 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:21.521846 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 463/924] 10.222.0.67 () {60 vars in 1144 bytes} [Thu Jun 30 16:14:21 2022] POST /identity/v3/endpoints => generated 347 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:14:22.697300 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6eab32-d79c-4ac5-810b-aad30204982c None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:22.698387 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6eab32-d79c-4ac5-810b-aad30204982c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:22.698960 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6eab32-d79c-4ac5-810b-aad30204982c None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:22.700869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 462/925] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:22.706873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a03610-79b3-4867-a272-37bea6566db1 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:22.707863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a03610-79b3-4867-a272-37bea6566db1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:22.708354 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a03610-79b3-4867-a272-37bea6566db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:22.728028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-16a03610-79b3-4867-a272-37bea6566db1 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:22.729743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-16a03610-79b3-4867-a272-37bea6566db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:22.743678 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:22.825117 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 464/926] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:22.849654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6eab32-d79c-4ac5-810b-aad30204982c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:22.907235 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf705d-3c02-4d33-ab80-18e3a9705686 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:22.907641 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf705d-3c02-4d33-ab80-18e3a9705686 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:22.907826 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf705d-3c02-4d33-ab80-18e3a9705686 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:22.909207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 465/927] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:14:22.915198 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac5dee-322b-4c64-bfa0-0c213c0fef21 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:22.915507 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac5dee-322b-4c64-bfa0-0c213c0fef21 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:22.915797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac5dee-322b-4c64-bfa0-0c213c0fef21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:22.934061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-35ac5dee-322b-4c64-bfa0-0c213c0fef21 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:22.935443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-35ac5dee-322b-4c64-bfa0-0c213c0fef21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:22.974745 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6eab32-d79c-4ac5-810b-aad30204982c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:23.014717 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4ee9c-cabe-42d3-a10a-bc8a11a41caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:23.015865 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4ee9c-cabe-42d3-a10a-bc8a11a41caa None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:23.016328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4ee9c-cabe-42d3-a10a-bc8a11a41caa None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:23.016748 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4ee9c-cabe-42d3-a10a-bc8a11a41caa None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:23.017553 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e4ee9c-cabe-42d3-a10a-bc8a11a41caa None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:23.019727 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e4ee9c-cabe-42d3-a10a-bc8a11a41caa None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:23.030215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 466/928] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:22 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 16:14:23.033385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-73e4ee9c-cabe-42d3-a10a-bc8a11a41caa None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 16:14:23.033385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:23.033385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:14:23.033385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:14:23.033385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:14:23.033385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:14:23.033385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:14:23.033385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:23.033385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 16:14:23.033385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:23.033385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:14:23.033385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:23.033385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:23.033385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:23.033385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:23.033385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:23.033385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:23.034598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:23.034598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:23.034598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:23.034598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:23.034598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:23.034598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:23.034598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:14:23.034598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:14:23.034598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:14:23.034598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:14:23.034598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:23.034598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:23.034598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:23.034598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:23.034598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:23.034598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:23.036120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:23.036120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:23.036120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:23.036120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:23.036120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:23.036120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:23.036120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:23.036120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:23.036120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:23.036120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:23.036120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:14:23.036120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:23.036120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 16:14:23.036120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:23.038516 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 463/929] 10.222.0.67 () {56 vars in 1126 bytes} [Thu Jun 30 16:14:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:23.044615 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac5dee-322b-4c64-bfa0-0c213c0fef21 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:23.060061 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4ee9c-cabe-42d3-a10a-bc8a11a41caa None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:23.122330 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:23.133105 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:23.159214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac5dee-322b-4c64-bfa0-0c213c0fef21 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:23.178632 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4ee9c-cabe-42d3-a10a-bc8a11a41caa None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:23.196076 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea2653-e3ce-47ec-998d-16a7bd6fb1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:23.196801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea2653-e3ce-47ec-998d-16a7bd6fb1ec None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:23.197081 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea2653-e3ce-47ec-998d-16a7bd6fb1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:23.197377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea2653-e3ce-47ec-998d-16a7bd6fb1ec None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:23.198070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea2653-e3ce-47ec-998d-16a7bd6fb1ec None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:23.198378 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea2653-e3ce-47ec-998d-16a7bd6fb1ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:23.199826 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea2653-e3ce-47ec-998d-16a7bd6fb1ec None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:23.204780 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 467/930] 10.222.0.67 () {56 vars in 1122 bytes} [Thu Jun 30 16:14:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:23.206046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91464177-9046-4af5-a822-8b97690cb62f None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:23.206332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91464177-9046-4af5-a822-8b97690cb62f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:23.207122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91464177-9046-4af5-a822-8b97690cb62f None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:23.208589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 468/931] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:23.213332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91464177-9046-4af5-a822-8b97690cb62f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:23.221489 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bcec38-fd47-49de-8794-e69d569deca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:23.222155 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bcec38-fd47-49de-8794-e69d569deca9 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:23.223098 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bcec38-fd47-49de-8794-e69d569deca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:23.223379 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bcec38-fd47-49de-8794-e69d569deca9 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:23.224323 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bcec38-fd47-49de-8794-e69d569deca9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:23.225832 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bcec38-fd47-49de-8794-e69d569deca9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:23.227171 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: WARNING keystone.server.flask.common [None req-46bcec38-fd47-49de-8794-e69d569deca9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 30 16:14:23.284169 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.sql.core [None req-46bcec38-fd47-49de-8794-e69d569deca9 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 16:14:23.284169 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 30 16:14:23.284169 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [parameters: {'id': 'bb264cc1aa2d4819b1c504dd13f199d2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 30 16:14:23.284169 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80297) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 16:14:23.302207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-46bcec38-fd47-49de-8794-e69d569deca9 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bb264cc1aa2d4819b1c504dd13f199d2.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bb264cc1aa2d4819b1c504dd13f199d2. Jun 30 16:14:23.302207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:23.302207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 16:14:23.302207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 16:14:23.302207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 16:14:23.302207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 16:14:23.302207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 16:14:23.302207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 16:14:23.302207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 16:14:23.302207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application conn.query(q) Jun 30 16:14:23.302207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 16:14:23.302207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 16:14:23.302207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 16:14:23.303120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result.read() Jun 30 16:14:23.303120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 16:14:23.303120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 16:14:23.303120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 16:14:23.303120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 16:14:23.303120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 16:14:23.303120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 16:14:23.303120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 16:14:23.303120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 16:14:23.303120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 16:14:23.303120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:23.303120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 16:14:23.303120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:23.303120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:23.303120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 16:14:23.303120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 16:14:23.303120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 30 16:14:23.303120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 30 16:14:23.304292 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 16:14:23.304292 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application next(self.gen) Jun 30 16:14:23.304292 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 16:14:23.304292 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application yield resource Jun 30 16:14:23.304292 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 16:14:23.304292 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application next(self.gen) Jun 30 16:14:23.304292 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 16:14:23.304292 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 16:14:23.304292 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 16:14:23.304292 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 16:14:23.304292 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 16:14:23.304292 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise self.value Jun 30 16:14:23.304292 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 16:14:23.304292 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 16:14:23.304292 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 16:14:23.304292 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application session.commit() Jun 30 16:14:23.304292 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 16:14:23.305729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 16:14:23.305729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 16:14:23.305729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 16:14:23.305729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 16:14:23.305729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.session.flush() Jun 30 16:14:23.305729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 16:14:23.305729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 16:14:23.305729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 16:14:23.305729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 16:14:23.305729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 16:14:23.305729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application compat.raise_( Jun 30 16:14:23.305729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 16:14:23.305729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception Jun 30 16:14:23.305729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 16:14:23.305729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 16:14:23.305729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 16:14:23.305729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 16:14:23.306621 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 16:14:23.306621 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 16:14:23.306621 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 16:14:23.306621 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 16:14:23.306621 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 30 16:14:23.306621 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 30 16:14:23.306621 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 16:14:23.306621 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 16:14:23.306621 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 16:14:23.306621 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 16:14:23.306621 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 16:14:23.306621 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 16:14:23.306621 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 16:14:23.306621 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 16:14:23.306621 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 16:14:23.308564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 16:14:23.308564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 16:14:23.308564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception Jun 30 16:14:23.308564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 16:14:23.308564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 16:14:23.308564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 16:14:23.308564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 16:14:23.308564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 16:14:23.308564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 16:14:23.308564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 16:14:23.308564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application conn.query(q) Jun 30 16:14:23.308564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 16:14:23.308564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 16:14:23.308564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 16:14:23.308564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result.read() Jun 30 16:14:23.308564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 16:14:23.308564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 16:14:23.309842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 16:14:23.309842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 16:14:23.309842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 16:14:23.309842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 16:14:23.309842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 16:14:23.309842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 16:14:23.309842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 16:14:23.309842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 30 16:14:23.309842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [parameters: {'id': 'bb264cc1aa2d4819b1c504dd13f199d2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 30 16:14:23.309842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 16:14:23.309842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:23.309842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:14:23.309842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:23.309842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:23.309842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 30 16:14:23.310888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 30 16:14:23.310888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 16:14:23.310888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 16:14:23.310888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 30 16:14:23.310888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:23.310888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:14:23.310888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:23.310888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:23.310888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:23.310888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:23.310888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:23.310888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:23.310888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:23.310888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:23.310888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:23.310888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:23.310888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:23.312032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:23.312032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 30 16:14:23.312032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 30 16:14:23.312032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:23.312032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:23.312032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 30 16:14:23.312032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 30 16:14:23.312032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bb264cc1aa2d4819b1c504dd13f199d2. Jun 30 16:14:23.312032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:23.312032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 464/932] 10.222.0.67 () {60 vars in 1140 bytes} [Thu Jun 30 16:14:23 2022] POST /identity/v3/projects => generated 262 bytes in 244 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:14:23.312032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd7800-f3e7-4071-bd10-bf9ee4e672bc None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:23.312032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd7800-f3e7-4071-bd10-bf9ee4e672bc None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:23.313526 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd7800-f3e7-4071-bd10-bf9ee4e672bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:23.330802 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7ccd7800-f3e7-4071-bd10-bf9ee4e672bc None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:23.332892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7ccd7800-f3e7-4071-bd10-bf9ee4e672bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:23.338439 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91464177-9046-4af5-a822-8b97690cb62f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:23.375945 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57363d-3254-47aa-9d62-9f2847193c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:23.376603 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57363d-3254-47aa-9d62-9f2847193c7d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:23.376888 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57363d-3254-47aa-9d62-9f2847193c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:23.377166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57363d-3254-47aa-9d62-9f2847193c7d None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:23.377877 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57363d-3254-47aa-9d62-9f2847193c7d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:23.378190 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57363d-3254-47aa-9d62-9f2847193c7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:23.380550 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57363d-3254-47aa-9d62-9f2847193c7d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:23.382706 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:23.386561 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 469/933] 10.222.0.67 () {56 vars in 1122 bytes} [Thu Jun 30 16:14:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:23.387368 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57363d-3254-47aa-9d62-9f2847193c7d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:23.419516 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 465/934] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:23.419516 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd7800-f3e7-4071-bd10-bf9ee4e672bc None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:23.474499 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:23.505728 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57363d-3254-47aa-9d62-9f2847193c7d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:23.535100 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd7800-f3e7-4071-bd10-bf9ee4e672bc None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:23.544312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2f00f-050b-41d9-916d-55f3d85cc143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:23.545784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2f00f-050b-41d9-916d-55f3d85cc143 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:23.546589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2f00f-050b-41d9-916d-55f3d85cc143 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:23.547350 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee2f00f-050b-41d9-916d-55f3d85cc143 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:23.556441 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2f00f-050b-41d9-916d-55f3d85cc143 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:23.558542 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee2f00f-050b-41d9-916d-55f3d85cc143 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:23.570336 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068c072-25b0-4deb-901d-05ce739a6fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:23.571188 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068c072-25b0-4deb-901d-05ce739a6fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:23.571703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068c072-25b0-4deb-901d-05ce739a6fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:23.572126 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068c072-25b0-4deb-901d-05ce739a6fe9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:23.573147 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068c072-25b0-4deb-901d-05ce739a6fe9 None admin] RBAC: Adding query filter params (interface=public, service_id=df984e94d11a4a8f922535655ebc0f52, region_id=RegionOne) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:23.573623 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068c072-25b0-4deb-901d-05ce739a6fe9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:23.574361 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8ee2f00f-050b-41d9-916d-55f3d85cc143 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:23.574361 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:23.574361 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:23.574361 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:23.574361 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:23.574361 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:23.574361 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:23.574361 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:23.574361 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:23.574361 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:23.574361 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:23.574361 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:23.574361 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:14:23.574361 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:14:23.574361 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:14:23.575433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:14:23.575433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:23.575433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:23.575433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:23.575433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:23.575433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:23.575433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:23.575433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:23.575433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:23.575433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:23.575433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:23.575433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:23.575433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:23.575433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:23.575433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:23.575433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:23.575433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:23.576571 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:14:23.576571 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:14:23.576571 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:14:23.576571 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:14:23.576571 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:14:23.576571 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:14:23.576571 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:23.576571 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:23.578153 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068c072-25b0-4deb-901d-05ce739a6fe9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:23.580629 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 470/935] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:14:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:23.581541 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2f00f-050b-41d9-916d-55f3d85cc143 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:23.588328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 466/936] 10.222.0.67 () {56 vars in 1251 bytes} [Thu Jun 30 16:14:23 2022] GET /identity/v3/endpoints?service_id=df984e94d11a4a8f922535655ebc0f52&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:23.589338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068c072-25b0-4deb-901d-05ce739a6fe9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:23.649901 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:23.694017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee2f00f-050b-41d9-916d-55f3d85cc143 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:23.705686 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068c072-25b0-4deb-901d-05ce739a6fe9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:23.716753 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:23.732663 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8901379-79b1-41b2-9b0c-6aa494ac0ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:23.737638 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8901379-79b1-41b2-9b0c-6aa494ac0ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:23.737638 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8901379-79b1-41b2-9b0c-6aa494ac0ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:23.737638 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8901379-79b1-41b2-9b0c-6aa494ac0ea5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:23.742745 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014944e6-2816-4373-8d2e-00078198cb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:23.743669 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014944e6-2816-4373-8d2e-00078198cb16 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:23.743956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014944e6-2816-4373-8d2e-00078198cb16 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:23.744237 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014944e6-2816-4373-8d2e-00078198cb16 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:23.744994 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014944e6-2816-4373-8d2e-00078198cb16 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:23.745300 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014944e6-2816-4373-8d2e-00078198cb16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:23.745791 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8901379-79b1-41b2-9b0c-6aa494ac0ea5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:14:23.746892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014944e6-2816-4373-8d2e-00078198cb16 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:23.754274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-c8901379-79b1-41b2-9b0c-6aa494ac0ea5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 16:14:23.754274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:23.754274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:23.754274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:23.754274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:23.754274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:23.754274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:23.754274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:23.754274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:23.754274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:23.754274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:23.754274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:23.754274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:14:23.754274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:14:23.754274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:14:23.754274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:14:23.755437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:14:23.755437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:14:23.755437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:14:23.755437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:14:23.755437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:23.755437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:23.755437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:23.755437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:23.755437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:23.755437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:23.755437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:23.755437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:23.755437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:23.755437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:23.755437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:23.755437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:23.756540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:23.756540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:23.756540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:23.756540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:23.756540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:14:23.756540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:14:23.756540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:14:23.756540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:14:23.756540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:14:23.756540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:14:23.756540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 16:14:23.756540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:23.756540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 471/937] 10.222.0.67 () {56 vars in 1120 bytes} [Thu Jun 30 16:14:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:23.756540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8901379-79b1-41b2-9b0c-6aa494ac0ea5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:23.765449 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 467/938] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 30 16:14:23 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:23.766746 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b3ab0-60b1-40a5-90c3-d9a9ef6b5515 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:23.767041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b3ab0-60b1-40a5-90c3-d9a9ef6b5515 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:23.767321 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245b3ab0-60b1-40a5-90c3-d9a9ef6b5515 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:23.773908 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 468/939] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:23.773908 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b3ab0-60b1-40a5-90c3-d9a9ef6b5515 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:23.878622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8901379-79b1-41b2-9b0c-6aa494ac0ea5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:23.897083 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245b3ab0-60b1-40a5-90c3-d9a9ef6b5515 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:23.904128 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:23.920143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac34d0-1a24-4411-a914-e9c8f9374e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:23.920865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac34d0-1a24-4411-a914-e9c8f9374e35 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:23.921096 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac34d0-1a24-4411-a914-e9c8f9374e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:23.922015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac34d0-1a24-4411-a914-e9c8f9374e35 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:23.922121 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac34d0-1a24-4411-a914-e9c8f9374e35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:23.923447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ac34d0-1a24-4411-a914-e9c8f9374e35 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:23.930445 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 472/940] 10.222.0.67 () {56 vars in 1087 bytes} [Thu Jun 30 16:14:23 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:14:23.931803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7c5bc-0029-4e2e-bf27-3c80c754a1cf None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:23.932090 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7c5bc-0029-4e2e-bf27-3c80c754a1cf None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:23.932368 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7c5bc-0029-4e2e-bf27-3c80c754a1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:23.948954 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff19f7f-0654-4104-8503-1b058aba7728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:23.950910 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff19f7f-0654-4104-8503-1b058aba7728 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:23.950910 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff19f7f-0654-4104-8503-1b058aba7728 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:23.950910 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff19f7f-0654-4104-8503-1b058aba7728 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:23.951224 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff19f7f-0654-4104-8503-1b058aba7728 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:23.951661 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff19f7f-0654-4104-8503-1b058aba7728 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:23.953160 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff19f7f-0654-4104-8503-1b058aba7728 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:23.958902 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fdd7c5bc-0029-4e2e-bf27-3c80c754a1cf None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:23.959271 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fdd7c5bc-0029-4e2e-bf27-3c80c754a1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:23.969980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 469/941] 10.222.0.67 () {56 vars in 1115 bytes} [Thu Jun 30 16:14:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:23.978646 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff19f7f-0654-4104-8503-1b058aba7728 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:24.025160 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:24.064231 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 473/942] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:23 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:24.099063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7c5bc-0029-4e2e-bf27-3c80c754a1cf None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:24.126679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff19f7f-0654-4104-8503-1b058aba7728 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:24.171564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6cef9-d39b-4c35-a86a-2972952f3531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:24.172344 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6cef9-d39b-4c35-a86a-2972952f3531 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:24.172626 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6cef9-d39b-4c35-a86a-2972952f3531 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:24.172898 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6cef9-d39b-4c35-a86a-2972952f3531 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:24.184997 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca6cef9-d39b-4c35-a86a-2972952f3531 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:24.186507 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca6cef9-d39b-4c35-a86a-2972952f3531 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:24.207889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-3ca6cef9-d39b-4c35-a86a-2972952f3531 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:24.207889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:24.207889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:24.207889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:24.207889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:24.207889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:24.207889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:24.207889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:24.207889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:24.207889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:24.207889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:24.207889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:24.207889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:24.207889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:24.207889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:24.208927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:24.208927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:24.208927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:24.208927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:24.208927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:24.208927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:24.208927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:24.208927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:24.208927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:24.208927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:24.208927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:24.208927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:24.208927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:24.208927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:24.208927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:24.208927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:24.208927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:24.209822 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:24.209822 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:24.209822 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:24.209822 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:24.209822 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 470/943] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:24.214945 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6cef9-d39b-4c35-a86a-2972952f3531 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:24.249938 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7c5bc-0029-4e2e-bf27-3c80c754a1cf None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:24.272981 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:24.301338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54686bbe-2020-4e0a-bcbe-3df5639a94be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:24.302999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54686bbe-2020-4e0a-bcbe-3df5639a94be None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:24.303462 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54686bbe-2020-4e0a-bcbe-3df5639a94be None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:24.303970 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54686bbe-2020-4e0a-bcbe-3df5639a94be None admin] PATH_INFO: `/v3/services/compute` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:24.305257 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54686bbe-2020-4e0a-bcbe-3df5639a94be None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:24.307563 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54686bbe-2020-4e0a-bcbe-3df5639a94be None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:24.313583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:24.325798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-54686bbe-2020-4e0a-bcbe-3df5639a94be None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 16:14:24.325798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:24.325798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:14:24.325798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:14:24.325798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:14:24.325798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:14:24.325798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:14:24.325798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:24.325798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 16:14:24.325798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:24.325798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:14:24.325798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:24.325798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:24.325798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:24.325798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:24.325798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:24.325798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:24.326835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:24.326835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:24.326835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:24.326835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:24.326835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:24.326835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:24.326835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:14:24.326835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:14:24.326835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:14:24.326835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:14:24.326835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:24.326835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:24.326835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:24.326835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:24.326835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:24.326835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:24.327717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:24.327717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:24.327717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:24.327717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:24.327717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:24.327717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:24.327717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:24.327717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:24.327717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:24.327717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:24.327717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:14:24.327717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:24.327717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 16:14:24.327717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:24.329363 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 474/944] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:14:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:14:24.335617 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54686bbe-2020-4e0a-bcbe-3df5639a94be None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:24.336354 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6cef9-d39b-4c35-a86a-2972952f3531 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:24.376124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22258c3-b148-4901-b1e0-7d7d12345152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:24.376937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22258c3-b148-4901-b1e0-7d7d12345152 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:24.377223 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22258c3-b148-4901-b1e0-7d7d12345152 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:24.377500 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22258c3-b148-4901-b1e0-7d7d12345152 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:24.378265 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22258c3-b148-4901-b1e0-7d7d12345152 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:24.379034 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22258c3-b148-4901-b1e0-7d7d12345152 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:24.380432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22258c3-b148-4901-b1e0-7d7d12345152 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:24.395358 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 471/945] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:24.401386 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22258c3-b148-4901-b1e0-7d7d12345152 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:24.458284 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54686bbe-2020-4e0a-bcbe-3df5639a94be None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:24.497006 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f864945-a6f4-4b82-95f1-1b27a5ea7ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:24.498122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f864945-a6f4-4b82-95f1-1b27a5ea7ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:24.498613 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f864945-a6f4-4b82-95f1-1b27a5ea7ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:24.499108 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f864945-a6f4-4b82-95f1-1b27a5ea7ba3 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:24.500063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f864945-a6f4-4b82-95f1-1b27a5ea7ba3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:24.500584 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f864945-a6f4-4b82-95f1-1b27a5ea7ba3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:24.502321 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f864945-a6f4-4b82-95f1-1b27a5ea7ba3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:24.509040 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 475/946] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 30 16:14:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:24.514263 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f864945-a6f4-4b82-95f1-1b27a5ea7ba3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:24.514842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22258c3-b148-4901-b1e0-7d7d12345152 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:24.560662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9bd53-6d9f-4dea-94b3-8fce04206653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:24.561531 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9bd53-6d9f-4dea-94b3-8fce04206653 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:24.561820 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9bd53-6d9f-4dea-94b3-8fce04206653 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:24.562098 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9bd53-6d9f-4dea-94b3-8fce04206653 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:24.570728 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9bd53-6d9f-4dea-94b3-8fce04206653 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:24.571317 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:24.572597 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9bd53-6d9f-4dea-94b3-8fce04206653 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:24.588724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-92c9bd53-6d9f-4dea-94b3-8fce04206653 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 16:14:24.588724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:24.588724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:24.588724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:24.588724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:24.588724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:24.588724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:24.588724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:24.588724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:24.588724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:24.588724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:24.588724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:24.588724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:14:24.588724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:14:24.588724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:14:24.588724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:14:24.589697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:24.589697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:24.589697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:14:24.589697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:24.589697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:14:24.589697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:24.589697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:24.589697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:24.589697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:24.589697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:24.589697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:24.589697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:24.589697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:24.589697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:24.589697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:24.589697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:24.589697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:24.589697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:24.590606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:24.590606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:24.590606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:14:24.590606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:14:24.590606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:14:24.590606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:14:24.590606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:14:24.590606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:14:24.590606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 16:14:24.590606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:24.594982 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 472/947] 10.222.0.67 () {56 vars in 1117 bytes} [Thu Jun 30 16:14:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:14:24.595886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9bd53-6d9f-4dea-94b3-8fce04206653 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:24.600030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:24.632319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f864945-a6f4-4b82-95f1-1b27a5ea7ba3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:24.670710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71adb981-2d2b-413c-be87-e928b12244b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:24.671715 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adb981-2d2b-413c-be87-e928b12244b5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:24.672177 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adb981-2d2b-413c-be87-e928b12244b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:24.672606 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adb981-2d2b-413c-be87-e928b12244b5 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:24.673461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71adb981-2d2b-413c-be87-e928b12244b5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:24.673916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71adb981-2d2b-413c-be87-e928b12244b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:24.676430 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71adb981-2d2b-413c-be87-e928b12244b5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:24.687987 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 476/948] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 30 16:14:24 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:24.705904 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9bd53-6d9f-4dea-94b3-8fce04206653 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:24.745522 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70243d61-5189-4d8b-b12e-8d9b432474fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:24.746218 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70243d61-5189-4d8b-b12e-8d9b432474fb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:24.746500 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70243d61-5189-4d8b-b12e-8d9b432474fb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:24.746775 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70243d61-5189-4d8b-b12e-8d9b432474fb None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:24.747681 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70243d61-5189-4d8b-b12e-8d9b432474fb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:24.747993 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70243d61-5189-4d8b-b12e-8d9b432474fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:24.749600 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70243d61-5189-4d8b-b12e-8d9b432474fb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:24.772856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 473/949] 10.222.0.67 () {56 vars in 1149 bytes} [Thu Jun 30 16:14:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:24.778653 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71adb981-2d2b-413c-be87-e928b12244b5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:24.891605 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71adb981-2d2b-413c-be87-e928b12244b5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:24.899095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:24.929602 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727735c3-e961-4347-a899-959585362a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:24.930404 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727735c3-e961-4347-a899-959585362a0f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:24.930686 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727735c3-e961-4347-a899-959585362a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:24.930961 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727735c3-e961-4347-a899-959585362a0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:24.941240 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727735c3-e961-4347-a899-959585362a0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:24.942697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727735c3-e961-4347-a899-959585362a0f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:24.962039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-727735c3-e961-4347-a899-959585362a0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:24.962039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:24.962039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:24.962039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:24.962039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:24.962039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:24.962039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:24.962039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:24.962039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:24.962039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:24.962039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:24.962039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:24.962039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:24.962039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:24.962039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:24.963109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:24.963109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:24.963109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:24.963109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:24.963109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:24.963109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:24.963109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:24.963109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:24.963109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:24.963109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:24.963109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:24.963109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:24.963109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:24.963109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:24.963109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:24.963109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:24.963109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:24.964038 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:24.964038 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:24.964038 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:24.964038 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:24.964038 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 477/950] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:24.969692 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70243d61-5189-4d8b-b12e-8d9b432474fb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:25.036547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:25.078898 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70243d61-5189-4d8b-b12e-8d9b432474fb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:25.116367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e830d-0090-4523-9be4-cf8a509c6523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:25.117037 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e830d-0090-4523-9be4-cf8a509c6523 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:25.117473 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e830d-0090-4523-9be4-cf8a509c6523 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:25.117750 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e830d-0090-4523-9be4-cf8a509c6523 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:25.118532 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e830d-0090-4523-9be4-cf8a509c6523 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:25.118888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e830d-0090-4523-9be4-cf8a509c6523 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:25.120293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e830d-0090-4523-9be4-cf8a509c6523 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:25.133386 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 474/951] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:25.138856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727735c3-e961-4347-a899-959585362a0f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:25.246382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727735c3-e961-4347-a899-959585362a0f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:25.283459 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164d413-acd7-4e50-9c8a-84b0d7690c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:25.284329 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164d413-acd7-4e50-9c8a-84b0d7690c08 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:25.284657 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164d413-acd7-4e50-9c8a-84b0d7690c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:25.284941 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164d413-acd7-4e50-9c8a-84b0d7690c08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:25.291857 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164d413-acd7-4e50-9c8a-84b0d7690c08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:25.293363 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164d413-acd7-4e50-9c8a-84b0d7690c08 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:25.307865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-6164d413-acd7-4e50-9c8a-84b0d7690c08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:25.307865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:25.307865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:25.307865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:25.307865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:25.307865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:25.307865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:25.307865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:25.307865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:25.307865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:25.307865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:25.307865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:25.307865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:14:25.307865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:14:25.307865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:14:25.308806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:14:25.308806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:25.308806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:25.308806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:25.308806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:25.308806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:25.308806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:25.308806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:25.308806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:25.308806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:25.308806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:25.308806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:25.308806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:25.308806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:25.308806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:25.308806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:25.308806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:25.309745 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:14:25.309745 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:14:25.309745 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:14:25.309745 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:14:25.309745 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:14:25.309745 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:14:25.309745 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:25.309745 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:25.309745 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 478/952] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:14:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:25.314985 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e830d-0090-4523-9be4-cf8a509c6523 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:25.419024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e830d-0090-4523-9be4-cf8a509c6523 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:25.444088 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:25.455925 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1b8709-54de-4434-b882-70d1ec4172d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:25.456732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b8709-54de-4434-b882-70d1ec4172d8 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:25.457163 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b8709-54de-4434-b882-70d1ec4172d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:25.457527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1b8709-54de-4434-b882-70d1ec4172d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:25.458317 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1b8709-54de-4434-b882-70d1ec4172d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:25.458625 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1b8709-54de-4434-b882-70d1ec4172d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:25.460156 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1b8709-54de-4434-b882-70d1ec4172d8 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:25.475837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 475/953] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:14:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:25.479375 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164d413-acd7-4e50-9c8a-84b0d7690c08 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:25.504946 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:25.587747 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164d413-acd7-4e50-9c8a-84b0d7690c08 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:25.622962 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7657b610-7269-4ffb-9afb-ccf3d17d004f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:25.623889 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657b610-7269-4ffb-9afb-ccf3d17d004f None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:25.624172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657b610-7269-4ffb-9afb-ccf3d17d004f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:25.624450 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657b610-7269-4ffb-9afb-ccf3d17d004f None admin] PATH_INFO: `/v3/projects/19ef92ebcce94b90bf39ad3a58772a1d/users/d49604bffa3d4e89a631eefa6d4e4f5d/roles/c2206d1398e84c28b5365a335a04e5ac` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:25.626789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7657b610-7269-4ffb-9afb-ccf3d17d004f None admin] RBAC: Authorizing `identity:create_grant(project_id=19ef92ebcce94b90bf39ad3a58772a1d, user_id=d49604bffa3d4e89a631eefa6d4e4f5d, role_id=c2206d1398e84c28b5365a335a04e5ac)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:25.628266 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7657b610-7269-4ffb-9afb-ccf3d17d004f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:25.708709 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 479/954] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 30 16:14:25 2022] PUT /identity/v3/projects/19ef92ebcce94b90bf39ad3a58772a1d/users/d49604bffa3d4e89a631eefa6d4e4f5d/roles/c2206d1398e84c28b5365a335a04e5ac => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:14:26.042044 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c81fa7-7c5d-44c9-9046-591e82b04891 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:26.042590 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c81fa7-7c5d-44c9-9046-591e82b04891 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:26.042590 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c81fa7-7c5d-44c9-9046-591e82b04891 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:26.044364 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 476/955] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:26.049953 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ff961-b44e-4a70-9165-b2e1e23e7d8b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:26.051530 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ff961-b44e-4a70-9165-b2e1e23e7d8b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:26.052042 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45ff961-b44e-4a70-9165-b2e1e23e7d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:26.069864 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d45ff961-b44e-4a70-9165-b2e1e23e7d8b None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:26.071459 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d45ff961-b44e-4a70-9165-b2e1e23e7d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:26.147796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 480/956] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 16:14:26.170269 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c81fa7-7c5d-44c9-9046-591e82b04891 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:26.188867 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb20581-0bb4-450e-a63b-d99ba2699692 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:26.189483 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb20581-0bb4-450e-a63b-d99ba2699692 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:26.190025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb20581-0bb4-450e-a63b-d99ba2699692 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:26.192027 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 481/957] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:26.199322 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1a2bb-1e73-4fe9-a5ba-05032fa85bce None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:26.200092 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1a2bb-1e73-4fe9-a5ba-05032fa85bce None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:26.200565 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1a2bb-1e73-4fe9-a5ba-05032fa85bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:26.223651 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d5b1a2bb-1e73-4fe9-a5ba-05032fa85bce None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:26.225400 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d5b1a2bb-1e73-4fe9-a5ba-05032fa85bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:26.228865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:26.291507 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c81fa7-7c5d-44c9-9046-591e82b04891 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:26.327500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 482/958] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 16:14:26.338719 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd118f1-a028-4437-9a53-6f4a073d85f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:26.339396 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd118f1-a028-4437-9a53-6f4a073d85f4 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:26.339737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd118f1-a028-4437-9a53-6f4a073d85f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:26.340017 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd118f1-a028-4437-9a53-6f4a073d85f4 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:26.340784 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd118f1-a028-4437-9a53-6f4a073d85f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:26.342262 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd118f1-a028-4437-9a53-6f4a073d85f4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:26.349873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1a2bb-1e73-4fe9-a5ba-05032fa85bce None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:26.398857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.sql.core [None req-fcd118f1-a028-4437-9a53-6f4a073d85f4 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 16:14:26.398857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 30 16:14:26.398857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [parameters: {'user_id': '750e382412d34053bbbb20dc0f96e347', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 30 16:14:26.398857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80297) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 16:14:26.407140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-fcd118f1-a028-4437-9a53-6f4a073d85f4 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 30 16:14:26.407140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:26.407140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 16:14:26.407140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 16:14:26.407140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 16:14:26.407140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 16:14:26.407140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 16:14:26.407140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 16:14:26.407140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 16:14:26.407140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application conn.query(q) Jun 30 16:14:26.407140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 16:14:26.407140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 16:14:26.407140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 16:14:26.407140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result.read() Jun 30 16:14:26.407140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 16:14:26.408078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 16:14:26.408078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 16:14:26.408078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 16:14:26.408078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 16:14:26.408078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 16:14:26.408078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 16:14:26.408078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 16:14:26.408078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 16:14:26.408078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:26.408078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 16:14:26.408078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:26.408078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:26.408078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 16:14:26.408078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 16:14:26.408078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 30 16:14:26.408078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 30 16:14:26.408078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 16:14:26.408078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application next(self.gen) Jun 30 16:14:26.409081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 16:14:26.409081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application yield resource Jun 30 16:14:26.409081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 16:14:26.409081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application next(self.gen) Jun 30 16:14:26.409081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 16:14:26.409081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 16:14:26.409081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 16:14:26.409081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 16:14:26.409081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 16:14:26.409081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise self.value Jun 30 16:14:26.409081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 16:14:26.409081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 16:14:26.409081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 16:14:26.409081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application session.commit() Jun 30 16:14:26.409081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 16:14:26.409081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 16:14:26.409081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 16:14:26.410305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 16:14:26.410305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 16:14:26.410305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.session.flush() Jun 30 16:14:26.410305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 16:14:26.410305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 16:14:26.410305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 16:14:26.410305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 16:14:26.410305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 16:14:26.410305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application compat.raise_( Jun 30 16:14:26.410305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 16:14:26.410305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception Jun 30 16:14:26.410305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 16:14:26.410305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 16:14:26.410305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 16:14:26.410305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 16:14:26.410305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 16:14:26.410305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 16:14:26.411195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 16:14:26.411195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 16:14:26.411195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 30 16:14:26.411195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 30 16:14:26.411195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 16:14:26.411195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 16:14:26.411195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 16:14:26.411195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 16:14:26.411195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 16:14:26.411195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 16:14:26.411195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 16:14:26.411195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 16:14:26.411195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 16:14:26.411195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 16:14:26.411195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 16:14:26.411195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception Jun 30 16:14:26.412095 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 16:14:26.412095 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 16:14:26.412095 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 16:14:26.412095 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 16:14:26.412095 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 16:14:26.412095 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 16:14:26.412095 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 16:14:26.412095 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application conn.query(q) Jun 30 16:14:26.412095 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 16:14:26.412095 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 16:14:26.412095 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 16:14:26.412095 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application result.read() Jun 30 16:14:26.412095 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 16:14:26.412095 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 16:14:26.412095 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 16:14:26.412095 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 16:14:26.413019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 16:14:26.413019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 16:14:26.413019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 16:14:26.413019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 16:14:26.413019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 16:14:26.413019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 30 16:14:26.413019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [parameters: {'user_id': '750e382412d34053bbbb20dc0f96e347', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 30 16:14:26.413019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 16:14:26.413019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:26.413019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:14:26.413019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:26.413019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:26.413019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:26.413019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:26.413019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:26.413825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:26.413825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:26.413825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:26.413825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:26.413825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:26.413825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:26.413825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:26.413825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 30 16:14:26.413825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 30 16:14:26.413825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:26.413825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:26.413825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:14:26.413825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:26.413825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:14:26.413825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:26.413825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 30 16:14:26.413825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 30 16:14:26.414776 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 30 16:14:26.414776 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 30 16:14:26.414776 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 16:14:26.414776 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 16:14:26.414776 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 30 16:14:26.414776 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:26.422802 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 477/959] 10.222.0.67 () {60 vars in 1131 bytes} [Thu Jun 30 16:14:26 2022] POST /identity/v3/users => generated 160 bytes in 253 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:26.424713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd118f1-a028-4437-9a53-6f4a073d85f4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:26.476345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1a2bb-1e73-4fe9-a5ba-05032fa85bce None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:26.479412 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:26.525374 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b4f15-48d6-40da-8a82-f438ab526144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:26.526198 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b4f15-48d6-40da-8a82-f438ab526144 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:26.526486 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b4f15-48d6-40da-8a82-f438ab526144 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:26.526761 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b4f15-48d6-40da-8a82-f438ab526144 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:26.527426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b4f15-48d6-40da-8a82-f438ab526144 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:26.528842 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b4f15-48d6-40da-8a82-f438ab526144 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:26.547252 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd118f1-a028-4437-9a53-6f4a073d85f4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:26.547451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-d15b4f15-48d6-40da-8a82-f438ab526144 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 16:14:26.547451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:26.547451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:14:26.547451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:14:26.547451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:14:26.547451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:14:26.547451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:14:26.547451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:26.547451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 16:14:26.547451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:26.547451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:14:26.547451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:26.547451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:26.547451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:26.547451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:26.547451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:26.547451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:26.548499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:26.548499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:26.548499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:26.548499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:26.548499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:26.548499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:26.548499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:14:26.548499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:14:26.548499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:14:26.548499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:14:26.548499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:26.548499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:26.548499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:26.548499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:26.548499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:26.548499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:26.549343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:26.549343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:26.549343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:26.549343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:26.549343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:26.549343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:26.549343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:26.549343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:26.549343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:26.549343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:26.549343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:14:26.549343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:26.549343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 16:14:26.549343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:26.549343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 483/960] 10.222.0.67 () {56 vars in 1126 bytes} [Thu Jun 30 16:14:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:14:26.554739 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b4f15-48d6-40da-8a82-f438ab526144 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:26.558961 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:26.590940 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0330af2f-b90f-4b7a-b805-1f4039d22c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:26.592406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330af2f-b90f-4b7a-b805-1f4039d22c86 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:26.593098 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330af2f-b90f-4b7a-b805-1f4039d22c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:26.593381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0330af2f-b90f-4b7a-b805-1f4039d22c86 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:26.604371 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0330af2f-b90f-4b7a-b805-1f4039d22c86 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:26.606031 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0330af2f-b90f-4b7a-b805-1f4039d22c86 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:26.623232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-0330af2f-b90f-4b7a-b805-1f4039d22c86 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 16:14:26.623232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:26.623232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:26.623232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:26.623232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:26.623232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:26.623232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:26.623232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:26.623232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:26.623232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:26.623232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:26.623232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:26.623232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:14:26.623232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:14:26.623232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:14:26.623232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:14:26.624407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:26.624407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:26.624407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:14:26.624407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:26.624407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:14:26.624407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:26.624407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:26.624407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:26.624407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:26.624407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:26.624407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:26.624407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:26.624407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:26.624407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:26.624407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:26.624407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:26.624407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:26.624407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:26.625346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:26.625346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:26.625346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:14:26.625346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:14:26.625346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:14:26.625346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:14:26.625346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:14:26.625346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:14:26.625346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 16:14:26.625346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:26.629614 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 478/961] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:14:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:26.630703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0330af2f-b90f-4b7a-b805-1f4039d22c86 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:26.695816 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b4f15-48d6-40da-8a82-f438ab526144 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:26.739322 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0330af2f-b90f-4b7a-b805-1f4039d22c86 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:26.742907 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:26.745917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee80cc-ae6f-4a25-9506-29e800cfeffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:26.746848 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee80cc-ae6f-4a25-9506-29e800cfeffe None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:26.747345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee80cc-ae6f-4a25-9506-29e800cfeffe None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:26.748267 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee80cc-ae6f-4a25-9506-29e800cfeffe None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:26.749357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee80cc-ae6f-4a25-9506-29e800cfeffe None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:26.753858 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee80cc-ae6f-4a25-9506-29e800cfeffe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:26.753858 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee80cc-ae6f-4a25-9506-29e800cfeffe None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:26.765800 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 484/962] 10.222.0.67 () {56 vars in 1122 bytes} [Thu Jun 30 16:14:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:26.769653 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee80cc-ae6f-4a25-9506-29e800cfeffe None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:26.782219 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a655d98-72a2-4429-9567-3f31eb992596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:26.782889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a655d98-72a2-4429-9567-3f31eb992596 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:26.783169 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a655d98-72a2-4429-9567-3f31eb992596 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:26.783443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a655d98-72a2-4429-9567-3f31eb992596 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:26.784264 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a655d98-72a2-4429-9567-3f31eb992596 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:26.785613 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a655d98-72a2-4429-9567-3f31eb992596 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:26.787092 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a655d98-72a2-4429-9567-3f31eb992596 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:26.843598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 479/963] 10.222.0.67 () {56 vars in 1101 bytes} [Thu Jun 30 16:14:26 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:26.864465 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:26.909437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee80cc-ae6f-4a25-9506-29e800cfeffe None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:26.921056 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5b291-0d19-458d-abf6-3eb6b964e105 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:26.921410 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5b291-0d19-458d-abf6-3eb6b964e105 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:26.921599 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5b291-0d19-458d-abf6-3eb6b964e105 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:26.923394 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 480/964] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:14:26.929460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced29e61-49aa-41b7-9838-d258f179a48a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:26.929747 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced29e61-49aa-41b7-9838-d258f179a48a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:26.930023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced29e61-49aa-41b7-9838-d258f179a48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:26.951945 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ced29e61-49aa-41b7-9838-d258f179a48a None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:26.953569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ced29e61-49aa-41b7-9838-d258f179a48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:26.962995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306b462-eee0-4f30-a916-f0865c232c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:26.964103 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306b462-eee0-4f30-a916-f0865c232c8f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:26.964620 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306b462-eee0-4f30-a916-f0865c232c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:26.965134 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2306b462-eee0-4f30-a916-f0865c232c8f None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:26.966264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306b462-eee0-4f30-a916-f0865c232c8f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:26.966786 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306b462-eee0-4f30-a916-f0865c232c8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:26.968783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2306b462-eee0-4f30-a916-f0865c232c8f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:26.982703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 485/965] 10.222.0.67 () {56 vars in 1122 bytes} [Thu Jun 30 16:14:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:26.990082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306b462-eee0-4f30-a916-f0865c232c8f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:27.086408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 481/966] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:26 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 16:14:27.114866 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced29e61-49aa-41b7-9838-d258f179a48a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:27.157802 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2306b462-eee0-4f30-a916-f0865c232c8f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:27.197888 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:27.204883 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67106b48-64ca-41ca-829b-112a32aec330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:27.205546 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67106b48-64ca-41ca-829b-112a32aec330 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:27.205826 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67106b48-64ca-41ca-829b-112a32aec330 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:27.206100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67106b48-64ca-41ca-829b-112a32aec330 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:27.207554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67106b48-64ca-41ca-829b-112a32aec330 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:27.209009 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67106b48-64ca-41ca-829b-112a32aec330 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:27.259252 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:27.268197 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced29e61-49aa-41b7-9838-d258f179a48a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:27.305062 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a856a81-c8a3-4d38-9d90-90016b92add9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:27.305729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a856a81-c8a3-4d38-9d90-90016b92add9 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:27.306008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a856a81-c8a3-4d38-9d90-90016b92add9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:27.306285 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a856a81-c8a3-4d38-9d90-90016b92add9 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:27.307007 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a856a81-c8a3-4d38-9d90-90016b92add9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:27.308438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a856a81-c8a3-4d38-9d90-90016b92add9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:27.359650 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 486/967] 10.222.0.67 () {60 vars in 1144 bytes} [Thu Jun 30 16:14:26 2022] POST /identity/v3/endpoints => generated 349 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:14:27.455823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 482/968] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 30 16:14:27 2022] POST /identity/v3/services => generated 242 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:14:27.861220 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633361c9-7886-424b-83d8-054502931938 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:27.861637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633361c9-7886-424b-83d8-054502931938 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:27.861830 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633361c9-7886-424b-83d8-054502931938 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:27.863406 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 487/969] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:27.868728 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ccba0b-0dbe-4099-9c5f-0f2332d1dc08 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:27.869026 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ccba0b-0dbe-4099-9c5f-0f2332d1dc08 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:27.869310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ccba0b-0dbe-4099-9c5f-0f2332d1dc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:27.887181 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-41ccba0b-0dbe-4099-9c5f-0f2332d1dc08 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:27.888519 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-41ccba0b-0dbe-4099-9c5f-0f2332d1dc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:27.969251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 483/970] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:27 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:27.995985 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633361c9-7886-424b-83d8-054502931938 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:28.107685 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633361c9-7886-424b-83d8-054502931938 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:28.145478 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeac172-be03-4fec-b1c1-bffa768446d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:28.146203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeac172-be03-4fec-b1c1-bffa768446d1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:28.146481 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeac172-be03-4fec-b1c1-bffa768446d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:28.146752 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeac172-be03-4fec-b1c1-bffa768446d1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:28.157358 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeac172-be03-4fec-b1c1-bffa768446d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:14:28.166707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-4eeac172-be03-4fec-b1c1-bffa768446d1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 16:14:28.166707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:28.166707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:28.166707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:28.166707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:28.166707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:28.166707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:28.166707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:28.166707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:28.166707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:28.166707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:28.166707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:28.166707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:14:28.166707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:14:28.166707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:14:28.166707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:14:28.173366 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:14:28.173366 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:14:28.173366 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:14:28.173366 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:14:28.173366 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:28.173366 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:28.173366 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:28.173366 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:28.173366 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:28.173366 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:28.173366 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:28.173366 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:28.173366 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:28.173366 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:28.173366 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:28.173366 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:28.174442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:28.174442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:28.174442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:28.174442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:28.174442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:14:28.174442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:14:28.174442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:14:28.174442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:14:28.174442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:14:28.174442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:14:28.174442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 16:14:28.174442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:28.174442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 488/971] 10.222.0.67 () {56 vars in 1120 bytes} [Thu Jun 30 16:14:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:28.177494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ccba0b-0dbe-4099-9c5f-0f2332d1dc08 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:28.259064 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:28.281466 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ccba0b-0dbe-4099-9c5f-0f2332d1dc08 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:28.321578 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2dc9c2-036e-474d-a378-b3e9f75c8966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:28.322509 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2dc9c2-036e-474d-a378-b3e9f75c8966 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:28.322944 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2dc9c2-036e-474d-a378-b3e9f75c8966 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:28.323374 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2dc9c2-036e-474d-a378-b3e9f75c8966 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:28.324288 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2dc9c2-036e-474d-a378-b3e9f75c8966 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:28.324747 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2dc9c2-036e-474d-a378-b3e9f75c8966 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:28.326286 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2dc9c2-036e-474d-a378-b3e9f75c8966 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:28.340641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 484/972] 10.222.0.67 () {56 vars in 1115 bytes} [Thu Jun 30 16:14:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:28.342464 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeac172-be03-4fec-b1c1-bffa768446d1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:28.391964 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:28.448788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeac172-be03-4fec-b1c1-bffa768446d1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:28.486710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006f6af-5748-44cc-936e-7470bb930bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:28.487957 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006f6af-5748-44cc-936e-7470bb930bea None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:28.488400 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006f6af-5748-44cc-936e-7470bb930bea None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:28.488829 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006f6af-5748-44cc-936e-7470bb930bea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:28.502127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006f6af-5748-44cc-936e-7470bb930bea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:28.503961 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006f6af-5748-44cc-936e-7470bb930bea None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:28.523216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-3006f6af-5748-44cc-936e-7470bb930bea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:28.523216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:28.523216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:28.523216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:28.523216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:28.523216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:28.523216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:28.523216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:28.523216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:28.523216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:28.523216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:28.523216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:28.523216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:28.523216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:28.523216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:28.524241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:28.524241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:28.524241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:28.524241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:28.524241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:28.524241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:28.524241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:28.524241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:28.524241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:28.524241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:28.524241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:28.524241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:28.524241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:28.524241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:28.524241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:28.524241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:28.524241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:28.529890 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:28.529890 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:28.529890 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:28.529890 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:28.529890 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 489/973] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:14:28.531658 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2dc9c2-036e-474d-a378-b3e9f75c8966 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:28.641046 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2dc9c2-036e-474d-a378-b3e9f75c8966 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:28.676890 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ba1f02-0e06-4051-a58d-dbb5a50c55dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:28.677871 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba1f02-0e06-4051-a58d-dbb5a50c55dc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:28.678329 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba1f02-0e06-4051-a58d-dbb5a50c55dc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:28.678794 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba1f02-0e06-4051-a58d-dbb5a50c55dc None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:28.679700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ba1f02-0e06-4051-a58d-dbb5a50c55dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:28.680180 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ba1f02-0e06-4051-a58d-dbb5a50c55dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:28.681698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ba1f02-0e06-4051-a58d-dbb5a50c55dc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:28.691806 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:28.695524 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 485/974] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:28.704902 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006f6af-5748-44cc-936e-7470bb930bea None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:28.804908 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:28.809410 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006f6af-5748-44cc-936e-7470bb930bea None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:28.845014 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d89738-41a6-4b75-837e-e3418f99f149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:28.845833 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d89738-41a6-4b75-837e-e3418f99f149 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:28.846117 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d89738-41a6-4b75-837e-e3418f99f149 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:28.846393 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d89738-41a6-4b75-837e-e3418f99f149 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:28.854610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d89738-41a6-4b75-837e-e3418f99f149 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:28.856141 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d89738-41a6-4b75-837e-e3418f99f149 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:28.871969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-d4d89738-41a6-4b75-837e-e3418f99f149 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 16:14:28.871969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:28.871969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:28.871969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:28.871969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:28.871969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:28.871969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:28.871969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:28.871969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:28.871969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:28.871969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:28.871969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:28.871969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:14:28.871969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:14:28.871969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:14:28.871969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:14:28.872948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:28.872948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:28.872948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:14:28.872948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:28.872948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:14:28.872948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:28.872948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:28.872948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:28.872948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:28.872948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:28.872948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:28.872948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:28.872948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:28.872948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:28.872948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:28.872948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:28.872948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:28.872948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:28.873967 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:28.873967 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:28.873967 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:14:28.873967 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:14:28.873967 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:14:28.873967 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:14:28.873967 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:14:28.873967 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:14:28.873967 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 16:14:28.873967 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:28.875858 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 490/975] 10.222.0.67 () {56 vars in 1117 bytes} [Thu Jun 30 16:14:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:28.879562 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ba1f02-0e06-4051-a58d-dbb5a50c55dc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:28.985797 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ba1f02-0e06-4051-a58d-dbb5a50c55dc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:29.030009 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90fd43-ace2-4d7c-9031-f3b5cba4165b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:29.030705 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90fd43-ace2-4d7c-9031-f3b5cba4165b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:29.030992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90fd43-ace2-4d7c-9031-f3b5cba4165b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:29.031293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d90fd43-ace2-4d7c-9031-f3b5cba4165b None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:29.032160 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d90fd43-ace2-4d7c-9031-f3b5cba4165b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:29.032470 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d90fd43-ace2-4d7c-9031-f3b5cba4165b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:29.033893 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d90fd43-ace2-4d7c-9031-f3b5cba4165b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:29.056770 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 486/976] 10.222.0.67 () {56 vars in 1149 bytes} [Thu Jun 30 16:14:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:29.058606 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d89738-41a6-4b75-837e-e3418f99f149 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:29.164792 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d89738-41a6-4b75-837e-e3418f99f149 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:29.201032 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0579b-44eb-4556-acad-985b615354ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:29.201699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0579b-44eb-4556-acad-985b615354ab None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:29.201957 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0579b-44eb-4556-acad-985b615354ab None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:29.202240 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0579b-44eb-4556-acad-985b615354ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:29.230374 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:29.275187 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0579b-44eb-4556-acad-985b615354ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:29.276611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0579b-44eb-4556-acad-985b615354ab None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:29.379647 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 491/977] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:14:29 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 16:14:29.383716 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90fd43-ace2-4d7c-9031-f3b5cba4165b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:29.440938 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271d55b-a937-40b1-90de-9836ea10a77b None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:29.441263 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271d55b-a937-40b1-90de-9836ea10a77b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:29.441534 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271d55b-a937-40b1-90de-9836ea10a77b None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:29.443059 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 492/978] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:14:29.446934 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:29.447849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ae02d-d4df-4c9e-bb2e-37433eb98e13 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:29.448140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ae02d-d4df-4c9e-bb2e-37433eb98e13 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:29.448548 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2ae02d-d4df-4c9e-bb2e-37433eb98e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:29.468387 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3e2ae02d-d4df-4c9e-bb2e-37433eb98e13 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:29.469581 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3e2ae02d-d4df-4c9e-bb2e-37433eb98e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:29.495010 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d90fd43-ace2-4d7c-9031-f3b5cba4165b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:29.530614 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:29.532159 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6b975-9a0a-4e15-9ab1-21b3d3add4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:29.532959 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6b975-9a0a-4e15-9ab1-21b3d3add4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:29.533268 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6b975-9a0a-4e15-9ab1-21b3d3add4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:29.533545 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6b975-9a0a-4e15-9ab1-21b3d3add4d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:29.544657 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6b975-9a0a-4e15-9ab1-21b3d3add4d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:29.544803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 493/979] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:29 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 16:14:29.546018 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ff8c0-3449-4591-aec9-1b2247b328f2 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:29.546294 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6b975-9a0a-4e15-9ab1-21b3d3add4d2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:29.546408 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ff8c0-3449-4591-aec9-1b2247b328f2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:29.546573 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5ff8c0-3449-4591-aec9-1b2247b328f2 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:29.547986 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 494/980] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:29.553450 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee135643-bb2a-4f5e-a392-c76e7ec8fe6e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:29.553738 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee135643-bb2a-4f5e-a392-c76e7ec8fe6e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:29.554016 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee135643-bb2a-4f5e-a392-c76e7ec8fe6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:29.568733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-21a6b975-9a0a-4e15-9ab1-21b3d3add4d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:29.568733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:29.568733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:29.568733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:29.568733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:29.568733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:29.568733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:29.568733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:29.568733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:29.568733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:29.568733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:29.568733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:29.568733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:29.568733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:29.568733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:29.569875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:29.569875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:29.569875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:29.569875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:29.569875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:29.569875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:29.569875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:29.569875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:29.569875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:29.569875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:29.569875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:29.569875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:29.569875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:29.569875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:29.569875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:29.569875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:29.569875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:29.570909 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:29.570909 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:29.570909 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:29.570909 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:29.570909 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 487/981] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:14:29.572312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ee135643-bb2a-4f5e-a392-c76e7ec8fe6e None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:29.572519 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6b975-9a0a-4e15-9ab1-21b3d3add4d2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:29.573501 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ee135643-bb2a-4f5e-a392-c76e7ec8fe6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:29.641072 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 495/982] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:29 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 16:14:29.641915 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee135643-bb2a-4f5e-a392-c76e7ec8fe6e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:29.678361 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6b975-9a0a-4e15-9ab1-21b3d3add4d2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:29.689133 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:29.714106 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518749e-a828-4353-be9e-d5fa2df8bffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:29.714921 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518749e-a828-4353-be9e-d5fa2df8bffb None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:29.715205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518749e-a828-4353-be9e-d5fa2df8bffb None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:29.715491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518749e-a828-4353-be9e-d5fa2df8bffb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:29.716157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518749e-a828-4353-be9e-d5fa2df8bffb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:29.717540 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518749e-a828-4353-be9e-d5fa2df8bffb None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:29.729662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-2518749e-a828-4353-be9e-d5fa2df8bffb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 16:14:29.729662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:29.729662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:14:29.729662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:14:29.729662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:14:29.729662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:14:29.729662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:14:29.729662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:29.729662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 16:14:29.729662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:29.729662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:14:29.729662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:29.729662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:29.729662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:29.729662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:29.729662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:29.729662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:29.730660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:29.730660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:29.730660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:29.730660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:29.730660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:29.730660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:29.730660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:14:29.730660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:14:29.730660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:14:29.730660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:14:29.730660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:29.730660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:29.730660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:29.730660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:29.730660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:29.730660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:29.731731 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:29.731731 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:29.731731 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:29.731731 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:29.731731 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:29.731731 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:29.731731 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:29.731731 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:29.731731 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:29.731731 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:29.731731 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:14:29.731731 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:29.731731 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 16:14:29.731731 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:29.731731 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 488/983] 10.222.0.67 () {56 vars in 1126 bytes} [Thu Jun 30 16:14:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:29.732522 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518749e-a828-4353-be9e-d5fa2df8bffb None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:29.747716 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee135643-bb2a-4f5e-a392-c76e7ec8fe6e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:29.767906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:29.786284 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e3e11-ad87-4906-a010-c5d6003e2159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:29.786982 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e3e11-ad87-4906-a010-c5d6003e2159 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:29.787270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e3e11-ad87-4906-a010-c5d6003e2159 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:29.787620 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e3e11-ad87-4906-a010-c5d6003e2159 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:29.788354 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e3e11-ad87-4906-a010-c5d6003e2159 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:29.788751 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e3e11-ad87-4906-a010-c5d6003e2159 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:29.790051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644e3e11-ad87-4906-a010-c5d6003e2159 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:29.802159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 496/984] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:29.803030 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e3e11-ad87-4906-a010-c5d6003e2159 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:29.839013 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518749e-a828-4353-be9e-d5fa2df8bffb None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:29.873657 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962fd55-402b-45dc-8008-89e7fafff57f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:29.874454 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962fd55-402b-45dc-8008-89e7fafff57f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:29.874741 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962fd55-402b-45dc-8008-89e7fafff57f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:29.875016 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962fd55-402b-45dc-8008-89e7fafff57f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:29.875695 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a962fd55-402b-45dc-8008-89e7fafff57f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:29.877066 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a962fd55-402b-45dc-8008-89e7fafff57f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:29.889724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a962fd55-402b-45dc-8008-89e7fafff57f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 16:14:29.889724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:29.889724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:14:29.889724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:14:29.889724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:14:29.889724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:14:29.889724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:14:29.889724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:29.889724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 16:14:29.889724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:29.889724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:14:29.889724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:29.889724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:29.889724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:29.889724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:29.889724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:29.889724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:29.890823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:29.890823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:29.890823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:29.890823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:29.890823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:29.890823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:29.890823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:14:29.890823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:14:29.890823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:14:29.890823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:14:29.890823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:29.890823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:29.890823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:29.890823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:29.890823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:29.890823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:29.891831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:29.891831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:29.891831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:29.891831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:29.891831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:29.891831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:29.891831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:29.891831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:29.891831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:29.891831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:29.891831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:14:29.891831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:29.891831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 16:14:29.891831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:29.891831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 489/985] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:14:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:29.892642 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962fd55-402b-45dc-8008-89e7fafff57f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:29.907175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e3e11-ad87-4906-a010-c5d6003e2159 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:29.934112 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:29.943248 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0096d1-63ee-440c-a9bf-9575da2ecb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:29.948625 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0096d1-63ee-440c-a9bf-9575da2ecb0d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:29.948625 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0096d1-63ee-440c-a9bf-9575da2ecb0d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:29.948625 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0096d1-63ee-440c-a9bf-9575da2ecb0d None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:29.948625 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0096d1-63ee-440c-a9bf-9575da2ecb0d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:29.948625 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0096d1-63ee-440c-a9bf-9575da2ecb0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:29.948625 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0096d1-63ee-440c-a9bf-9575da2ecb0d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:29.951523 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 497/986] 10.222.0.67 () {56 vars in 1122 bytes} [Thu Jun 30 16:14:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:29.951523 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0096d1-63ee-440c-a9bf-9575da2ecb0d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:29.997553 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:30.001690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962fd55-402b-45dc-8008-89e7fafff57f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:30.039659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83003b14-c999-4d51-8214-35b9e0292698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:30.040360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83003b14-c999-4d51-8214-35b9e0292698 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:30.040635 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83003b14-c999-4d51-8214-35b9e0292698 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:30.040902 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83003b14-c999-4d51-8214-35b9e0292698 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:30.047757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83003b14-c999-4d51-8214-35b9e0292698 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:30.049196 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83003b14-c999-4d51-8214-35b9e0292698 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:30.056055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0096d1-63ee-440c-a9bf-9575da2ecb0d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:30.063847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-83003b14-c999-4d51-8214-35b9e0292698 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:30.063847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:30.063847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:30.063847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:30.063847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:30.063847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:30.063847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:30.063847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:30.063847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:30.063847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:30.063847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:30.063847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:30.063847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:14:30.063847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:14:30.063847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:14:30.064938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:14:30.064938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:30.064938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:30.064938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:30.064938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:30.064938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:30.064938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:30.064938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:30.064938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:30.064938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:30.064938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:30.064938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:30.064938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:30.064938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:30.064938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:30.064938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:30.064938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:30.066992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:14:30.066992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:14:30.066992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:14:30.066992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:14:30.066992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:14:30.066992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:14:30.066992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:30.066992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:30.066992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 490/987] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:14:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:30.066992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83003b14-c999-4d51-8214-35b9e0292698 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:30.094444 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce5487-d0c0-4cf3-a5a6-82433a55d926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:30.095121 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce5487-d0c0-4cf3-a5a6-82433a55d926 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:30.095399 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce5487-d0c0-4cf3-a5a6-82433a55d926 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:30.095706 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce5487-d0c0-4cf3-a5a6-82433a55d926 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:30.096424 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dce5487-d0c0-4cf3-a5a6-82433a55d926 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:30.096767 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dce5487-d0c0-4cf3-a5a6-82433a55d926 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:30.098258 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dce5487-d0c0-4cf3-a5a6-82433a55d926 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:30.103324 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 498/988] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 30 16:14:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:30.104152 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce5487-d0c0-4cf3-a5a6-82433a55d926 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:30.172297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:30.176801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83003b14-c999-4d51-8214-35b9e0292698 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:30.206647 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce5487-d0c0-4cf3-a5a6-82433a55d926 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:30.212594 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54656333-b950-4093-948e-423c547935ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:30.213264 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54656333-b950-4093-948e-423c547935ea None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:30.213541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54656333-b950-4093-948e-423c547935ea None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:30.213825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54656333-b950-4093-948e-423c547935ea None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:30.213973 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:30.214594 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54656333-b950-4093-948e-423c547935ea None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:30.214826 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54656333-b950-4093-948e-423c547935ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:30.216222 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54656333-b950-4093-948e-423c547935ea None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:30.221224 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 491/989] 10.222.0.67 () {56 vars in 1122 bytes} [Thu Jun 30 16:14:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:30.222578 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54656333-b950-4093-948e-423c547935ea None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:30.244091 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b601b51-be0f-4a63-95b3-9f9a3ffa109e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:30.244751 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b601b51-be0f-4a63-95b3-9f9a3ffa109e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:30.245038 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b601b51-be0f-4a63-95b3-9f9a3ffa109e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:30.245319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b601b51-be0f-4a63-95b3-9f9a3ffa109e None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:30.246104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b601b51-be0f-4a63-95b3-9f9a3ffa109e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:30.246417 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b601b51-be0f-4a63-95b3-9f9a3ffa109e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:30.247993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b601b51-be0f-4a63-95b3-9f9a3ffa109e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:30.259570 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 499/990] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:14:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:30.260405 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b601b51-be0f-4a63-95b3-9f9a3ffa109e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:30.325179 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54656333-b950-4093-948e-423c547935ea None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:30.359913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2c6ef-26e2-4cee-8fb7-0a4c5c40e932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:30.360573 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2c6ef-26e2-4cee-8fb7-0a4c5c40e932 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:30.360858 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2c6ef-26e2-4cee-8fb7-0a4c5c40e932 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:30.361137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2c6ef-26e2-4cee-8fb7-0a4c5c40e932 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:30.361296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b601b51-be0f-4a63-95b3-9f9a3ffa109e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:30.361839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2c6ef-26e2-4cee-8fb7-0a4c5c40e932 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:30.362147 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2c6ef-26e2-4cee-8fb7-0a4c5c40e932 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:30.364124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2c6ef-26e2-4cee-8fb7-0a4c5c40e932 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:30.373764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 492/991] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 30 16:14:30 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:30.373764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2c6ef-26e2-4cee-8fb7-0a4c5c40e932 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:30.396779 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:30.397621 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9b23b-830b-484f-b440-11c0ac11671d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:30.397831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9b23b-830b-484f-b440-11c0ac11671d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:30.398048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9b23b-830b-484f-b440-11c0ac11671d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:30.398332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9b23b-830b-484f-b440-11c0ac11671d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:30.399124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb9b23b-830b-484f-b440-11c0ac11671d None admin] RBAC: Adding query filter params (interface=admin, service_id=df984e94d11a4a8f922535655ebc0f52, region_id=RegionOne) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:30.399439 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb9b23b-830b-484f-b440-11c0ac11671d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:30.400839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb9b23b-830b-484f-b440-11c0ac11671d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:30.405842 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 500/992] 10.222.0.67 () {56 vars in 1249 bytes} [Thu Jun 30 16:14:30 2022] GET /identity/v3/endpoints?service_id=df984e94d11a4a8f922535655ebc0f52&interface=admin®ion_id=RegionOne => generated 193 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:30.411919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9b23b-830b-484f-b440-11c0ac11671d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:30.437818 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:30.471084 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2c6ef-26e2-4cee-8fb7-0a4c5c40e932 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:30.506938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a21c6-9861-48c2-89d0-161841653ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:30.507616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a21c6-9861-48c2-89d0-161841653ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:30.507902 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a21c6-9861-48c2-89d0-161841653ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:30.508178 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a21c6-9861-48c2-89d0-161841653ae7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:30.508273 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9b23b-830b-484f-b440-11c0ac11671d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:30.509004 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a21c6-9861-48c2-89d0-161841653ae7 None admin] RBAC: Adding query filter params (role.id=c2206d1398e84c28b5365a335a04e5ac, scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d, user.id=d49604bffa3d4e89a631eefa6d4e4f5d) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:30.509315 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a21c6-9861-48c2-89d0-161841653ae7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:30.510759 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a21c6-9861-48c2-89d0-161841653ae7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:30.531504 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 493/993] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:14:30 2022] GET /identity/v3/role_assignments?user.id=d49604bffa3d4e89a631eefa6d4e4f5d&scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d&role.id=c2206d1398e84c28b5365a335a04e5ac => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:30.531504 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a21c6-9861-48c2-89d0-161841653ae7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:30.547761 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8a081-ac82-4f5d-b136-f80aa0437600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:30.548447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8a081-ac82-4f5d-b136-f80aa0437600 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:30.548764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8a081-ac82-4f5d-b136-f80aa0437600 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:30.549046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8a081-ac82-4f5d-b136-f80aa0437600 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:30.550134 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db8a081-ac82-4f5d-b136-f80aa0437600 None admin] RBAC: Adding query filter params (interface=public, service_id=c01e00f4782c4f6b813b59361ac33c6d, region_id=RegionOne) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:30.551256 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db8a081-ac82-4f5d-b136-f80aa0437600 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:30.552740 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db8a081-ac82-4f5d-b136-f80aa0437600 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:30.559034 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 501/994] 10.222.0.67 () {56 vars in 1251 bytes} [Thu Jun 30 16:14:30 2022] GET /identity/v3/endpoints?service_id=c01e00f4782c4f6b813b59361ac33c6d&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:30.564540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8a081-ac82-4f5d-b136-f80aa0437600 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:30.623134 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:30.634028 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a21c6-9861-48c2-89d0-161841653ae7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:30.674779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:30.679668 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7066a4c-3c94-41ab-910f-c8c4e57aa3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:30.680581 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7066a4c-3c94-41ab-910f-c8c4e57aa3ad None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:30.680869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7066a4c-3c94-41ab-910f-c8c4e57aa3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:30.681151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7066a4c-3c94-41ab-910f-c8c4e57aa3ad None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:30.681838 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7066a4c-3c94-41ab-910f-c8c4e57aa3ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:30.683269 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7066a4c-3c94-41ab-910f-c8c4e57aa3ad None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:30.684169 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8a081-ac82-4f5d-b136-f80aa0437600 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:30.691011 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 494/995] 10.222.0.67 () {56 vars in 1087 bytes} [Thu Jun 30 16:14:30 2022] GET /identity/v3/services => generated 1727 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:14:30.725751 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496973d3-61dd-4b06-a06b-829c53e08c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:30.726551 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496973d3-61dd-4b06-a06b-829c53e08c2e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:30.726904 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496973d3-61dd-4b06-a06b-829c53e08c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:30.727185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496973d3-61dd-4b06-a06b-829c53e08c2e None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:30.728008 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496973d3-61dd-4b06-a06b-829c53e08c2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:30.729462 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496973d3-61dd-4b06-a06b-829c53e08c2e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:30.736754 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 502/996] 10.222.0.67 () {56 vars in 1087 bytes} [Thu Jun 30 16:14:30 2022] GET /identity/v3/services => generated 1727 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:14:32.612107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67121c74-b308-41f8-a44f-6e572e9d5263 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:32.613154 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67121c74-b308-41f8-a44f-6e572e9d5263 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:32.613154 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67121c74-b308-41f8-a44f-6e572e9d5263 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:32.615515 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 495/997] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:32.619345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8bd22-32ac-4e51-aa59-7b19a0573fd8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:32.619807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8bd22-32ac-4e51-aa59-7b19a0573fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:32.619976 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8bd22-32ac-4e51-aa59-7b19a0573fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:32.637883 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-05c8bd22-32ac-4e51-aa59-7b19a0573fd8 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:32.639064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-05c8bd22-32ac-4e51-aa59-7b19a0573fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:32.713415 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 503/998] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:32.735428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67121c74-b308-41f8-a44f-6e572e9d5263 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:32.784686 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace89cd4-4d7a-47b4-928c-d993e9dc9764 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:32.785022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace89cd4-4d7a-47b4-928c-d993e9dc9764 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:32.785752 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace89cd4-4d7a-47b4-928c-d993e9dc9764 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:32.787520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 504/999] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:14:32.792811 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a44e7-a759-466d-a927-0912af129870 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:32.793112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a44e7-a759-466d-a927-0912af129870 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:32.793397 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a44e7-a759-466d-a927-0912af129870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:32.812223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-437a44e7-a759-466d-a927-0912af129870 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:32.813911 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-437a44e7-a759-466d-a927-0912af129870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:32.848993 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67121c74-b308-41f8-a44f-6e572e9d5263 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:32.884638 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 505/1000] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 16:14:32.884820 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807f14e-a71a-4dcf-8937-834f76265e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:32.885536 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807f14e-a71a-4dcf-8937-834f76265e70 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:32.885819 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807f14e-a71a-4dcf-8937-834f76265e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:32.885938 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f64579-185e-4a1a-99eb-fa4cebc5b0e7 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:32.886091 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7807f14e-a71a-4dcf-8937-834f76265e70 None admin] PATH_INFO: `/v3/services/image` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:32.886244 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f64579-185e-4a1a-99eb-fa4cebc5b0e7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:32.886481 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f64579-185e-4a1a-99eb-fa4cebc5b0e7 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:32.886747 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7807f14e-a71a-4dcf-8937-834f76265e70 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:32.887967 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 506/1001] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:32.888197 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7807f14e-a71a-4dcf-8937-834f76265e70 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:32.893137 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c04d24-6c1f-4a3a-a795-2771b4975aa3 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:32.893501 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c04d24-6c1f-4a3a-a795-2771b4975aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:32.894415 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c04d24-6c1f-4a3a-a795-2771b4975aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:32.901103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-7807f14e-a71a-4dcf-8937-834f76265e70 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 16:14:32.901103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:32.901103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:14:32.901103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:14:32.901103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:14:32.901103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:14:32.901103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:14:32.901103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:32.901103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 16:14:32.901103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:32.901103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:14:32.901103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:32.901103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:32.901103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:32.901103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:32.901103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:32.901103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:32.902189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:32.902189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:32.902189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:32.902189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:32.902189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:32.902189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:32.902189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:14:32.902189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:14:32.902189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:14:32.902189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:14:32.902189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:32.902189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:32.902189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:32.902189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:32.902189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:32.902189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:32.903130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:32.903130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:32.903130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:32.903130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:32.903130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:32.903130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:32.903130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:32.903130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:32.903130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:32.903130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:32.903130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:14:32.903130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:32.903130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 16:14:32.903130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:32.903130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 496/1002] 10.222.0.67 () {56 vars in 1105 bytes} [Thu Jun 30 16:14:32 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:32.907368 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807f14e-a71a-4dcf-8937-834f76265e70 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:32.915410 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-51c04d24-6c1f-4a3a-a795-2771b4975aa3 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:32.916672 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-51c04d24-6c1f-4a3a-a795-2771b4975aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:32.928206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:32.930919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:32.984915 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 507/1003] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:32 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 16:14:32.985681 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c04d24-6c1f-4a3a-a795-2771b4975aa3 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:33.023408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7807f14e-a71a-4dcf-8937-834f76265e70 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:33.059455 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312a901-93ba-4a2b-8596-c7014833d8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:33.060256 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312a901-93ba-4a2b-8596-c7014833d8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:33.060650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312a901-93ba-4a2b-8596-c7014833d8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:33.061100 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312a901-93ba-4a2b-8596-c7014833d8e3 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:33.061902 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312a901-93ba-4a2b-8596-c7014833d8e3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:33.062414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312a901-93ba-4a2b-8596-c7014833d8e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:33.064193 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312a901-93ba-4a2b-8596-c7014833d8e3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:33.070595 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 497/1004] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:33.071232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312a901-93ba-4a2b-8596-c7014833d8e3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:33.100754 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c04d24-6c1f-4a3a-a795-2771b4975aa3 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:33.136329 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114801d6-206a-469e-96a8-e5dc097dc39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:33.137052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114801d6-206a-469e-96a8-e5dc097dc39f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:33.137330 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114801d6-206a-469e-96a8-e5dc097dc39f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:33.137606 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114801d6-206a-469e-96a8-e5dc097dc39f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:33.138377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114801d6-206a-469e-96a8-e5dc097dc39f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:33.139791 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114801d6-206a-469e-96a8-e5dc097dc39f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:33.152127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-114801d6-206a-469e-96a8-e5dc097dc39f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 16:14:33.152127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:33.152127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:14:33.152127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:14:33.152127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:14:33.152127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:14:33.152127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:14:33.152127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:33.152127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 16:14:33.152127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:33.152127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:14:33.152127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:33.152127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:33.152127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:33.152127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:33.152127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:33.152127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:33.153211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:33.153211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:33.153211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:33.153211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:33.153211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:33.153211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:33.153211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:14:33.153211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:14:33.153211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:14:33.153211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:14:33.153211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:33.153211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:33.153211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:33.153211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:33.153211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:33.153211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:33.154048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:33.154048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:33.154048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:33.154048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:33.154048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:33.154048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:33.154048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:33.154048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:33.154048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:33.154048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:33.154048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:14:33.154048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:33.154048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 16:14:33.154048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:33.154048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 508/1005] 10.222.0.67 () {56 vars in 1126 bytes} [Thu Jun 30 16:14:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:33.154844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114801d6-206a-469e-96a8-e5dc097dc39f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:33.163528 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:33.170140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:33.182209 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312a901-93ba-4a2b-8596-c7014833d8e3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:33.216226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b9d13-6694-4830-9a4a-0f7a57319547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:33.217023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b9d13-6694-4830-9a4a-0f7a57319547 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:33.217310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b9d13-6694-4830-9a4a-0f7a57319547 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:33.217579 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b9d13-6694-4830-9a4a-0f7a57319547 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:33.218226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b9d13-6694-4830-9a4a-0f7a57319547 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:33.219787 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b9d13-6694-4830-9a4a-0f7a57319547 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:33.232569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e60b9d13-6694-4830-9a4a-0f7a57319547 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 16:14:33.232569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:33.232569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:14:33.232569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:14:33.232569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:14:33.232569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:14:33.232569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:14:33.232569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:33.232569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 16:14:33.232569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:33.232569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:14:33.232569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:33.232569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:33.232569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:33.232569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:33.232569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:33.232569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:33.233641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:33.233641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:33.233641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:33.233641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:33.233641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:33.233641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:33.233641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:14:33.233641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:14:33.233641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:14:33.233641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:14:33.233641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:33.233641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:33.233641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:33.233641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:33.233641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:33.233641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:33.234463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:33.234463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:33.234463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:33.234463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:33.234463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:33.234463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:33.234463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:33.234463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:33.234463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:33.234463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:33.234463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:14:33.234463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:33.234463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 16:14:33.234463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:33.234463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 498/1006] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:14:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:33.235250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b9d13-6694-4830-9a4a-0f7a57319547 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:33.257852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114801d6-206a-469e-96a8-e5dc097dc39f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:33.292517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6966137-e21d-4557-ad09-0f17eb25b5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:33.293192 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6966137-e21d-4557-ad09-0f17eb25b5c7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:33.293475 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6966137-e21d-4557-ad09-0f17eb25b5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:33.293754 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6966137-e21d-4557-ad09-0f17eb25b5c7 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:33.294460 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6966137-e21d-4557-ad09-0f17eb25b5c7 None admin] RBAC: Adding query filter params (type=image) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:33.294771 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6966137-e21d-4557-ad09-0f17eb25b5c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:33.296178 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6966137-e21d-4557-ad09-0f17eb25b5c7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:33.301342 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 509/1007] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:33 2022] GET /identity/v3/services?type=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:33.302118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6966137-e21d-4557-ad09-0f17eb25b5c7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:33.352746 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b9d13-6694-4830-9a4a-0f7a57319547 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:33.392333 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:33.395378 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb483ad-cf0f-4d5c-a0a7-6a5fb8c99318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:33.396092 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb483ad-cf0f-4d5c-a0a7-6a5fb8c99318 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:33.396380 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb483ad-cf0f-4d5c-a0a7-6a5fb8c99318 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:33.396755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb483ad-cf0f-4d5c-a0a7-6a5fb8c99318 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:33.397722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb483ad-cf0f-4d5c-a0a7-6a5fb8c99318 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:33.398036 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb483ad-cf0f-4d5c-a0a7-6a5fb8c99318 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:33.399921 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb483ad-cf0f-4d5c-a0a7-6a5fb8c99318 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:33.411297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 499/1008] 10.222.0.67 () {56 vars in 1122 bytes} [Thu Jun 30 16:14:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:33.413612 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb483ad-cf0f-4d5c-a0a7-6a5fb8c99318 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:33.428504 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:33.429259 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6966137-e21d-4557-ad09-0f17eb25b5c7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:33.493960 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0061645-860e-4cd3-828b-9e6be7b93dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:33.495449 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0061645-860e-4cd3-828b-9e6be7b93dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:33.496020 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0061645-860e-4cd3-828b-9e6be7b93dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:33.496661 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0061645-860e-4cd3-828b-9e6be7b93dc8 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:33.498015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0061645-860e-4cd3-828b-9e6be7b93dc8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:33.499109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0061645-860e-4cd3-828b-9e6be7b93dc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:33.503048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0061645-860e-4cd3-828b-9e6be7b93dc8 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:33.509837 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 510/1009] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 30 16:14:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:33.515405 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0061645-860e-4cd3-828b-9e6be7b93dc8 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:33.547950 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb483ad-cf0f-4d5c-a0a7-6a5fb8c99318 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:33.584690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e5287e-56d7-477b-a433-0026e745ba92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:33.585420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e5287e-56d7-477b-a433-0026e745ba92 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:33.585771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e5287e-56d7-477b-a433-0026e745ba92 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:33.586051 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e5287e-56d7-477b-a433-0026e745ba92 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:33.586759 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e5287e-56d7-477b-a433-0026e745ba92 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:33.587068 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e5287e-56d7-477b-a433-0026e745ba92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:33.588593 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e5287e-56d7-477b-a433-0026e745ba92 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:33.593884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 500/1010] 10.222.0.67 () {56 vars in 1122 bytes} [Thu Jun 30 16:14:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:33.601579 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e5287e-56d7-477b-a433-0026e745ba92 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:33.632894 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0061645-860e-4cd3-828b-9e6be7b93dc8 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:33.671569 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1983b02-4f0c-4a82-8b1e-9cad419991d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:33.672164 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1983b02-4f0c-4a82-8b1e-9cad419991d5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:33.672453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1983b02-4f0c-4a82-8b1e-9cad419991d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:33.672732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1983b02-4f0c-4a82-8b1e-9cad419991d5 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:33.673452 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1983b02-4f0c-4a82-8b1e-9cad419991d5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:33.673762 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1983b02-4f0c-4a82-8b1e-9cad419991d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:33.675185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1983b02-4f0c-4a82-8b1e-9cad419991d5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:33.680439 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 511/1011] 10.222.0.67 () {56 vars in 1112 bytes} [Thu Jun 30 16:14:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:33.686015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1983b02-4f0c-4a82-8b1e-9cad419991d5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:33.686408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:33.689722 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:33.712962 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e5287e-56d7-477b-a433-0026e745ba92 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:33.749104 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10b2a1c-7f06-4fc4-9b8b-6e80744b8a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:33.749773 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b2a1c-7f06-4fc4-9b8b-6e80744b8a84 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:33.750080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b2a1c-7f06-4fc4-9b8b-6e80744b8a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:33.750358 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10b2a1c-7f06-4fc4-9b8b-6e80744b8a84 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:33.751101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b2a1c-7f06-4fc4-9b8b-6e80744b8a84 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:33.752541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10b2a1c-7f06-4fc4-9b8b-6e80744b8a84 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:33.791644 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1983b02-4f0c-4a82-8b1e-9cad419991d5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:33.835508 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 501/1012] 10.222.0.67 () {60 vars in 1144 bytes} [Thu Jun 30 16:14:33 2022] POST /identity/v3/endpoints => generated 325 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:14:33.847212 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c09c8e-df6c-4b9e-a2f4-b04ebed5968a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:33.848759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c09c8e-df6c-4b9e-a2f4-b04ebed5968a None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:33.849112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c09c8e-df6c-4b9e-a2f4-b04ebed5968a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:33.849461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c09c8e-df6c-4b9e-a2f4-b04ebed5968a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:33.850302 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c09c8e-df6c-4b9e-a2f4-b04ebed5968a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:33.851768 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c09c8e-df6c-4b9e-a2f4-b04ebed5968a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:33.949161 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 512/1013] 10.222.0.67 () {60 vars in 1144 bytes} [Thu Jun 30 16:14:33 2022] POST /identity/v3/endpoints => generated 334 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:14:35.403707 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fe49c-54d5-41f9-8c4a-f4f2bac5966f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:35.404237 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fe49c-54d5-41f9-8c4a-f4f2bac5966f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:35.404346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06fe49c-54d5-41f9-8c4a-f4f2bac5966f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:35.405771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 502/1014] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:35.411699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fde88-59e4-4927-bada-a1573a2156a5 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:35.412121 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fde88-59e4-4927-bada-a1573a2156a5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:35.412290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688fde88-59e4-4927-bada-a1573a2156a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:35.431991 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-688fde88-59e4-4927-bada-a1573a2156a5 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:35.433286 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-688fde88-59e4-4927-bada-a1573a2156a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:35.490138 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:35.513827 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 513/1015] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:35 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 16:14:35.538083 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06fe49c-54d5-41f9-8c4a-f4f2bac5966f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:35.645754 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06fe49c-54d5-41f9-8c4a-f4f2bac5966f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:35.680936 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372f791-c673-4d2c-82b2-3ce7d8e67aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:35.681517 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372f791-c673-4d2c-82b2-3ce7d8e67aa9 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:35.681907 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372f791-c673-4d2c-82b2-3ce7d8e67aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:35.682192 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372f791-c673-4d2c-82b2-3ce7d8e67aa9 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:35.682920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7372f791-c673-4d2c-82b2-3ce7d8e67aa9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:35.684305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7372f791-c673-4d2c-82b2-3ce7d8e67aa9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:35.686448 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:35.758960 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 503/1016] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 30 16:14:35 2022] POST /identity/v3/services => generated 242 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:14:35.960758 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d77bde-d5ef-42f9-8491-e096b201896a None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:35.961065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d77bde-d5ef-42f9-8491-e096b201896a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:35.961261 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d77bde-d5ef-42f9-8491-e096b201896a None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:35.962740 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 514/1017] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:35.967797 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30aeb0-37c9-41c2-9fdc-848c8cf59852 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:35.968083 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30aeb0-37c9-41c2-9fdc-848c8cf59852 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:35.968356 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb30aeb0-37c9-41c2-9fdc-848c8cf59852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:35.987819 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-eb30aeb0-37c9-41c2-9fdc-848c8cf59852 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:35.988686 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-eb30aeb0-37c9-41c2-9fdc-848c8cf59852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:36.070293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 504/1018] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:35 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:36.094261 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d77bde-d5ef-42f9-8491-e096b201896a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:36.122132 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d0fc2-c7f6-4c33-8c18-f1c971ca8ede None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:36.122432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d0fc2-c7f6-4c33-8c18-f1c971ca8ede None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:36.122692 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d0fc2-c7f6-4c33-8c18-f1c971ca8ede None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:36.124322 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 505/1019] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:14:36.130233 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f62de-f30d-4cf4-8e40-2a4b23620839 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:36.130522 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f62de-f30d-4cf4-8e40-2a4b23620839 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:36.130799 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f62de-f30d-4cf4-8e40-2a4b23620839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:36.148709 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d08f62de-f30d-4cf4-8e40-2a4b23620839 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:36.149883 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d08f62de-f30d-4cf4-8e40-2a4b23620839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:36.209824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d77bde-d5ef-42f9-8491-e096b201896a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:36.223630 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 506/1020] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:36 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:36.244521 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f62de-f30d-4cf4-8e40-2a4b23620839 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:36.248967 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8deafa0-793d-4770-a6df-8d5de06782a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:36.249751 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8deafa0-793d-4770-a6df-8d5de06782a9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:36.250031 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8deafa0-793d-4770-a6df-8d5de06782a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:36.250298 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8deafa0-793d-4770-a6df-8d5de06782a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:36.251498 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8deafa0-793d-4770-a6df-8d5de06782a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:36.254200 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8deafa0-793d-4770-a6df-8d5de06782a9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:36.256014 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 515/1021] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:36.261585 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8deafa0-793d-4770-a6df-8d5de06782a9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:36.296843 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:36.328973 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:36.348365 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f62de-f30d-4cf4-8e40-2a4b23620839 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:36.383788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8deafa0-793d-4770-a6df-8d5de06782a9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:36.386181 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741506b-da64-4ee9-9fcd-bd8810ad4ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:36.386993 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741506b-da64-4ee9-9fcd-bd8810ad4ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:36.387272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741506b-da64-4ee9-9fcd-bd8810ad4ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:36.387576 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741506b-da64-4ee9-9fcd-bd8810ad4ea7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:36.399399 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741506b-da64-4ee9-9fcd-bd8810ad4ea7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:14:36.407589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-c741506b-da64-4ee9-9fcd-bd8810ad4ea7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 16:14:36.407589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:36.407589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:36.407589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:36.407589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:36.407589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:36.407589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:36.407589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:36.407589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:36.407589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:36.407589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:36.407589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:36.407589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:14:36.407589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:14:36.407589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:14:36.407589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:14:36.409172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:14:36.409172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:14:36.409172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:14:36.409172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:14:36.409172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:36.409172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:36.409172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:36.409172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:36.409172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:36.409172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:36.409172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:36.409172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:36.409172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:36.409172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:36.409172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:36.409172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:36.410041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:36.410041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:36.410041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:36.410041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:36.410041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:14:36.410041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:14:36.410041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:14:36.410041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:14:36.410041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:14:36.410041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:14:36.410041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 16:14:36.410041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:36.410041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 507/1022] 10.222.0.67 () {56 vars in 1120 bytes} [Thu Jun 30 16:14:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:36.415582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741506b-da64-4ee9-9fcd-bd8810ad4ea7 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:36.424596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf8040-a18f-435e-8ab8-96cbb9ec35c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:36.425266 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf8040-a18f-435e-8ab8-96cbb9ec35c4 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:36.425656 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf8040-a18f-435e-8ab8-96cbb9ec35c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:36.425938 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf8040-a18f-435e-8ab8-96cbb9ec35c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:36.426654 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcf8040-a18f-435e-8ab8-96cbb9ec35c4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:36.428129 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcf8040-a18f-435e-8ab8-96cbb9ec35c4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:36.519980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741506b-da64-4ee9-9fcd-bd8810ad4ea7 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:36.531024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:36.539744 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 516/1023] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 30 16:14:36 2022] POST /identity/v3/projects => generated 311 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:14:36.563523 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d816b-0b84-45dd-b22f-fdbf8f291d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:36.563892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d816b-0b84-45dd-b22f-fdbf8f291d8e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:36.565643 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d816b-0b84-45dd-b22f-fdbf8f291d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:36.565643 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d816b-0b84-45dd-b22f-fdbf8f291d8e None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:36.565807 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d816b-0b84-45dd-b22f-fdbf8f291d8e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:36.566210 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d816b-0b84-45dd-b22f-fdbf8f291d8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:36.567928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d816b-0b84-45dd-b22f-fdbf8f291d8e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:36.578467 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 508/1024] 10.222.0.67 () {56 vars in 1115 bytes} [Thu Jun 30 16:14:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:36.584520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf8040-a18f-435e-8ab8-96cbb9ec35c4 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:36.700931 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf8040-a18f-435e-8ab8-96cbb9ec35c4 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:36.702991 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:36.736809 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c592b-0cd1-424c-84b4-c6c559155947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:36.738588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c592b-0cd1-424c-84b4-c6c559155947 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:36.738878 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c592b-0cd1-424c-84b4-c6c559155947 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:36.739235 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3c592b-0cd1-424c-84b4-c6c559155947 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:36.749953 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c592b-0cd1-424c-84b4-c6c559155947 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:36.751424 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3c592b-0cd1-424c-84b4-c6c559155947 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:36.772388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8a3c592b-0cd1-424c-84b4-c6c559155947 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:36.772388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:36.772388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:36.772388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:36.772388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:36.772388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:36.772388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:36.772388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:36.772388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:36.772388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:36.772388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:36.772388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:36.772388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:36.772388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:36.772388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:36.773367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:36.773367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:36.773367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:36.773367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:36.773367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:36.773367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:36.773367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:36.773367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:36.773367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:36.773367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:36.773367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:36.773367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:36.773367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:36.773367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:36.773367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:36.773367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:36.773367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:36.774537 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:36.774537 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:36.774537 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:36.774537 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:36.774537 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 517/1025] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:36.781095 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d816b-0b84-45dd-b22f-fdbf8f291d8e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:36.931304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d816b-0b84-45dd-b22f-fdbf8f291d8e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:36.992043 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca94eb67-c560-4944-8215-1a17751fa245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:36.995651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94eb67-c560-4944-8215-1a17751fa245 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:36.995651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94eb67-c560-4944-8215-1a17751fa245 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:36.995651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94eb67-c560-4944-8215-1a17751fa245 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:36.995995 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca94eb67-c560-4944-8215-1a17751fa245 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:36.999671 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca94eb67-c560-4944-8215-1a17751fa245 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:36.999671 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca94eb67-c560-4944-8215-1a17751fa245 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:37.017062 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 509/1026] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:37.023320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c592b-0cd1-424c-84b4-c6c559155947 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:37.129488 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3c592b-0cd1-424c-84b4-c6c559155947 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:37.165849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c040a58-8ee8-4f5a-a6d0-15041e141504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:37.166647 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c040a58-8ee8-4f5a-a6d0-15041e141504 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:37.166933 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c040a58-8ee8-4f5a-a6d0-15041e141504 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:37.167213 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c040a58-8ee8-4f5a-a6d0-15041e141504 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:37.174374 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c040a58-8ee8-4f5a-a6d0-15041e141504 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:37.176380 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c040a58-8ee8-4f5a-a6d0-15041e141504 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:37.192023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-6c040a58-8ee8-4f5a-a6d0-15041e141504 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 16:14:37.192023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:37.192023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:37.192023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:37.192023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:37.192023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:37.192023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:37.192023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:37.192023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:37.192023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:37.192023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:37.192023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:37.192023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:14:37.192023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:14:37.192023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:14:37.192023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:14:37.193052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:37.193052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:37.193052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:14:37.193052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:37.193052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:14:37.193052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:37.193052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:37.193052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:37.193052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:37.193052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:37.193052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:37.193052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:37.193052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:37.193052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:37.193052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:37.193052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:37.193052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:37.193052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:37.194012 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:37.194012 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:37.194012 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:14:37.194012 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:14:37.194012 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:14:37.194012 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:14:37.194012 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:14:37.194012 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:14:37.194012 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 16:14:37.194012 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:37.195632 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 518/1027] 10.222.0.67 () {56 vars in 1093 bytes} [Thu Jun 30 16:14:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:14:37.198967 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca94eb67-c560-4944-8215-1a17751fa245 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:37.233471 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:37.305616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca94eb67-c560-4944-8215-1a17751fa245 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:37.340662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2cebb-5698-43c7-ad94-3caa6c227909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:37.341333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2cebb-5698-43c7-ad94-3caa6c227909 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:37.341616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2cebb-5698-43c7-ad94-3caa6c227909 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:37.341892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2cebb-5698-43c7-ad94-3caa6c227909 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:37.342723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2cebb-5698-43c7-ad94-3caa6c227909 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:37.343035 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2cebb-5698-43c7-ad94-3caa6c227909 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:37.344605 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2cebb-5698-43c7-ad94-3caa6c227909 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:37.367741 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 510/1028] 10.222.0.67 () {56 vars in 1133 bytes} [Thu Jun 30 16:14:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:37.369029 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c040a58-8ee8-4f5a-a6d0-15041e141504 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:37.386122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:37.474482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c040a58-8ee8-4f5a-a6d0-15041e141504 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:37.508919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d4791-2fe4-404c-b682-12f943cc94f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:37.509568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d4791-2fe4-404c-b682-12f943cc94f0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:37.509853 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d4791-2fe4-404c-b682-12f943cc94f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:37.510131 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d4791-2fe4-404c-b682-12f943cc94f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:37.578565 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698d4791-2fe4-404c-b682-12f943cc94f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:37.580061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698d4791-2fe4-404c-b682-12f943cc94f0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:37.595071 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:37.681589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 519/1029] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:14:37 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 16:14:37.687086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2cebb-5698-43c7-ad94-3caa6c227909 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:37.783079 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:37.793457 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2cebb-5698-43c7-ad94-3caa6c227909 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:37.832119 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec64390-fab4-4e44-8166-fccda79a77a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:37.832847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec64390-fab4-4e44-8166-fccda79a77a0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:37.833132 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec64390-fab4-4e44-8166-fccda79a77a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:37.833406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec64390-fab4-4e44-8166-fccda79a77a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:37.844369 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec64390-fab4-4e44-8166-fccda79a77a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:37.845871 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec64390-fab4-4e44-8166-fccda79a77a0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:37.864291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-fec64390-fab4-4e44-8166-fccda79a77a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:37.864291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:37.864291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:37.864291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:37.864291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:37.864291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:37.864291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:37.864291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:37.864291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:37.864291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:37.864291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:37.864291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:37.864291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:37.864291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:37.864291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:37.865293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:37.865293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:37.865293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:37.865293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:37.865293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:37.865293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:37.865293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:37.865293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:37.865293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:37.865293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:37.865293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:37.865293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:37.865293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:37.865293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:37.865293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:37.865293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:37.865293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:37.870881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:37.870881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:37.870881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:37.870881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:37.870881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 511/1030] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:14:37.871175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3e372-22e4-4097-9e05-ce6c50bd3f7d None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:37.871175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3e372-22e4-4097-9e05-ce6c50bd3f7d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:37.871175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3e372-22e4-4097-9e05-ce6c50bd3f7d None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:37.871175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 520/1031] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:37.871697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec64390-fab4-4e44-8166-fccda79a77a0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:37.875211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474a15c-716c-414b-9d97-169aa0ca7e96 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:37.875573 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474a15c-716c-414b-9d97-169aa0ca7e96 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:37.875864 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474a15c-716c-414b-9d97-169aa0ca7e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:37.895022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d474a15c-716c-414b-9d97-169aa0ca7e96 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:37.896800 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d474a15c-716c-414b-9d97-169aa0ca7e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:37.981221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec64390-fab4-4e44-8166-fccda79a77a0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:37.985483 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 521/1032] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:37 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 16:14:38.010560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d474a15c-716c-414b-9d97-169aa0ca7e96 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:38.025678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d0d8b-c9c6-46ab-9a22-c8cdfd932882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:38.026348 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d0d8b-c9c6-46ab-9a22-c8cdfd932882 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:38.026636 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d0d8b-c9c6-46ab-9a22-c8cdfd932882 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:38.026914 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d0d8b-c9c6-46ab-9a22-c8cdfd932882 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:38.027655 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d0d8b-c9c6-46ab-9a22-c8cdfd932882 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:38.027964 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d0d8b-c9c6-46ab-9a22-c8cdfd932882 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:38.029322 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d0d8b-c9c6-46ab-9a22-c8cdfd932882 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:38.041735 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 512/1033] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:38.047391 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d0d8b-c9c6-46ab-9a22-c8cdfd932882 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:38.051000 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:38.061650 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:38.119428 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d474a15c-716c-414b-9d97-169aa0ca7e96 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:38.153123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d0d8b-c9c6-46ab-9a22-c8cdfd932882 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:38.158898 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464bee5-58e4-4a32-b43c-5aab4dce8fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:38.159901 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464bee5-58e4-4a32-b43c-5aab4dce8fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:38.160855 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464bee5-58e4-4a32-b43c-5aab4dce8fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:38.161315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464bee5-58e4-4a32-b43c-5aab4dce8fa7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:38.162126 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464bee5-58e4-4a32-b43c-5aab4dce8fa7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:38.164560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464bee5-58e4-4a32-b43c-5aab4dce8fa7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:38.181649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8464bee5-58e4-4a32-b43c-5aab4dce8fa7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 16:14:38.181649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:38.181649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:14:38.181649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:14:38.181649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:14:38.181649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:14:38.181649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:14:38.181649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:38.181649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 16:14:38.181649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:38.181649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:14:38.181649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:38.181649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:38.181649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:38.181649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:38.181649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:38.181649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:38.182794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:38.182794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:38.182794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:38.182794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:38.182794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:38.182794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:38.182794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:14:38.182794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:14:38.182794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:14:38.182794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:14:38.182794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:38.182794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:38.182794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:38.182794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:38.182794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:38.182794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:38.183942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:38.183942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:38.183942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:38.183942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:38.183942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:38.183942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:38.183942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:38.183942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:38.183942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:38.183942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:38.183942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:14:38.183942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:38.183942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 16:14:38.183942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:38.184724 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 522/1034] 10.222.0.67 () {56 vars in 1105 bytes} [Thu Jun 30 16:14:38 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:38.189524 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be6470-361b-4805-818f-7dbee2c180d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:38.190649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464bee5-58e4-4a32-b43c-5aab4dce8fa7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:38.190861 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be6470-361b-4805-818f-7dbee2c180d9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:38.191147 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be6470-361b-4805-818f-7dbee2c180d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:38.191425 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be6470-361b-4805-818f-7dbee2c180d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:38.199359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be6470-361b-4805-818f-7dbee2c180d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:38.201297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2be6470-361b-4805-818f-7dbee2c180d9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:38.217585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f2be6470-361b-4805-818f-7dbee2c180d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:38.217585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:38.217585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:38.217585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:38.217585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:38.217585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:38.217585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:38.217585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:38.217585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:38.217585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:38.217585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:38.217585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:38.217585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:14:38.217585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:14:38.217585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:14:38.218544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:14:38.218544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:38.218544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:38.218544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:38.218544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:38.218544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:38.218544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:38.218544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:38.218544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:38.218544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:38.218544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:38.218544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:38.218544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:38.218544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:38.218544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:38.218544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:38.218544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:38.219440 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:14:38.219440 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:14:38.219440 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:14:38.219440 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:14:38.219440 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:14:38.219440 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:14:38.219440 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:38.219440 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:38.219440 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 513/1035] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:14:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:14:38.225012 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be6470-361b-4805-818f-7dbee2c180d9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:38.295615 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:38.309242 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464bee5-58e4-4a32-b43c-5aab4dce8fa7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:38.319274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:38.337650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2be6470-361b-4805-818f-7dbee2c180d9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:38.344803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc886119-9395-4f70-bc11-6990bba16379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:38.345465 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc886119-9395-4f70-bc11-6990bba16379 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:38.345751 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc886119-9395-4f70-bc11-6990bba16379 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:38.346026 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc886119-9395-4f70-bc11-6990bba16379 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:38.347278 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc886119-9395-4f70-bc11-6990bba16379 None admin] RBAC: Adding query filter params (name=image) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:38.347761 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc886119-9395-4f70-bc11-6990bba16379 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:38.349144 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc886119-9395-4f70-bc11-6990bba16379 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:38.354382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 523/1036] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:38.359755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc886119-9395-4f70-bc11-6990bba16379 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:38.385759 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e3fad-1992-499f-a55b-1077555ae575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:38.386591 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e3fad-1992-499f-a55b-1077555ae575 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:38.386875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e3fad-1992-499f-a55b-1077555ae575 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:38.387606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e3fad-1992-499f-a55b-1077555ae575 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:38.388677 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e3fad-1992-499f-a55b-1077555ae575 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:38.388989 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e3fad-1992-499f-a55b-1077555ae575 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:38.391435 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e3fad-1992-499f-a55b-1077555ae575 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:38.408719 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 514/1037] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:14:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:38.414798 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e3fad-1992-499f-a55b-1077555ae575 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:38.474011 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc886119-9395-4f70-bc11-6990bba16379 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:38.512955 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01965af-1525-46dd-afc5-1eda26f624b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:38.514529 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01965af-1525-46dd-afc5-1eda26f624b5 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:38.515174 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01965af-1525-46dd-afc5-1eda26f624b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:38.515690 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01965af-1525-46dd-afc5-1eda26f624b5 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:38.516570 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01965af-1525-46dd-afc5-1eda26f624b5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:38.517172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01965af-1525-46dd-afc5-1eda26f624b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:38.519373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01965af-1525-46dd-afc5-1eda26f624b5 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:38.521447 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e3fad-1992-499f-a55b-1077555ae575 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:38.525508 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 524/1038] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:38.530948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01965af-1525-46dd-afc5-1eda26f624b5 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:38.547404 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:38.559273 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa0b77-1117-4760-aa9d-9750e30c9ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:38.561802 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa0b77-1117-4760-aa9d-9750e30c9ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:38.561802 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa0b77-1117-4760-aa9d-9750e30c9ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:38.561802 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa0b77-1117-4760-aa9d-9750e30c9ae2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:38.561802 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa0b77-1117-4760-aa9d-9750e30c9ae2 None admin] RBAC: Adding query filter params (role.id=c2206d1398e84c28b5365a335a04e5ac, scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d, user.id=71cca3708386459197abf6c67505bc5f) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:38.561802 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa0b77-1117-4760-aa9d-9750e30c9ae2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:38.563316 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa0b77-1117-4760-aa9d-9750e30c9ae2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:38.569773 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 515/1039] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:14:38 2022] GET /identity/v3/role_assignments?user.id=71cca3708386459197abf6c67505bc5f&scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d&role.id=c2206d1398e84c28b5365a335a04e5ac => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:38.572454 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:38.646527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01965af-1525-46dd-afc5-1eda26f624b5 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:38.684108 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d5235-aa85-4858-9962-cd754ccb41e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:38.684795 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d5235-aa85-4858-9962-cd754ccb41e7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:38.685082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d5235-aa85-4858-9962-cd754ccb41e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:38.685361 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d5235-aa85-4858-9962-cd754ccb41e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:38.686217 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d5235-aa85-4858-9962-cd754ccb41e7 None admin] RBAC: Adding query filter params (interface=public, service_id=8404fa0968974522abe1d61727ef8df5, region_id=RegionOne) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:38.686535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d5235-aa85-4858-9962-cd754ccb41e7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:38.688045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d5235-aa85-4858-9962-cd754ccb41e7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:38.699446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 525/1040] 10.222.0.67 () {56 vars in 1251 bytes} [Thu Jun 30 16:14:38 2022] GET /identity/v3/endpoints?service_id=8404fa0968974522abe1d61727ef8df5&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:38.704823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa0b77-1117-4760-aa9d-9750e30c9ae2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:38.806955 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abc87d-5b31-496f-8c1e-33f4b3f7395b None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:38.807437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abc87d-5b31-496f-8c1e-33f4b3f7395b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:38.807679 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abc87d-5b31-496f-8c1e-33f4b3f7395b None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:38.809061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 526/1041] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:14:38.811253 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa0b77-1117-4760-aa9d-9750e30c9ae2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:38.816199 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b15bf-4d75-4654-b2b7-85a9630c795f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:38.816492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b15bf-4d75-4654-b2b7-85a9630c795f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:38.816772 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b15bf-4d75-4654-b2b7-85a9630c795f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:38.834952 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fa5b15bf-4d75-4654-b2b7-85a9630c795f None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:38.836272 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fa5b15bf-4d75-4654-b2b7-85a9630c795f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:38.847192 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb47e1-76a9-49d5-81aa-71a780b8df64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:38.847794 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb47e1-76a9-49d5-81aa-71a780b8df64 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:38.848076 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb47e1-76a9-49d5-81aa-71a780b8df64 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:38.848525 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb47e1-76a9-49d5-81aa-71a780b8df64 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:38.849022 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb47e1-76a9-49d5-81aa-71a780b8df64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:38.850385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb47e1-76a9-49d5-81aa-71a780b8df64 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:38.857401 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 516/1042] 10.222.0.67 () {56 vars in 1087 bytes} [Thu Jun 30 16:14:38 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:14:38.919594 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 527/1043] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:38 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 16:14:38.942851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb47e1-76a9-49d5-81aa-71a780b8df64 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:39.011372 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:39.066598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb47e1-76a9-49d5-81aa-71a780b8df64 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:39.100635 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51636add-15ad-40ce-a375-1b0341162e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:39.101431 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51636add-15ad-40ce-a375-1b0341162e14 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:39.101713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51636add-15ad-40ce-a375-1b0341162e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:39.101986 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51636add-15ad-40ce-a375-1b0341162e14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:39.103164 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51636add-15ad-40ce-a375-1b0341162e14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:39.104622 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51636add-15ad-40ce-a375-1b0341162e14 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:39.106508 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 517/1044] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:39.112721 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b15bf-4d75-4654-b2b7-85a9630c795f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:39.143512 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:39.214850 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b15bf-4d75-4654-b2b7-85a9630c795f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:39.250934 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291d1fe-0ad0-4b42-8f97-524ff1dec6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:39.251642 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291d1fe-0ad0-4b42-8f97-524ff1dec6e3 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:39.251926 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291d1fe-0ad0-4b42-8f97-524ff1dec6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:39.252200 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291d1fe-0ad0-4b42-8f97-524ff1dec6e3 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:39.252973 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291d1fe-0ad0-4b42-8f97-524ff1dec6e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:39.254443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a291d1fe-0ad0-4b42-8f97-524ff1dec6e3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:39.353444 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 528/1045] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 30 16:14:39 2022] POST /identity/v3/users => generated 285 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:14:40.674091 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b7804-e51f-4cd7-b21e-5349f0f99162 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:40.675277 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b7804-e51f-4cd7-b21e-5349f0f99162 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:40.675277 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57b7804-e51f-4cd7-b21e-5349f0f99162 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:40.676646 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 518/1046] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:40.682164 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20111a4-e01e-405a-8df8-72b41328dbc3 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:40.682637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20111a4-e01e-405a-8df8-72b41328dbc3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:40.682725 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20111a4-e01e-405a-8df8-72b41328dbc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:40.699605 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f20111a4-e01e-405a-8df8-72b41328dbc3 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:40.700797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f20111a4-e01e-405a-8df8-72b41328dbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:40.769565 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 529/1047] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:40.793918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b7804-e51f-4cd7-b21e-5349f0f99162 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:40.900414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57b7804-e51f-4cd7-b21e-5349f0f99162 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:40.918309 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:40.935855 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad76f2-da1a-4788-baa6-b5076a6a5d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:40.936595 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad76f2-da1a-4788-baa6-b5076a6a5d73 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:40.936875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad76f2-da1a-4788-baa6-b5076a6a5d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:40.937152 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad76f2-da1a-4788-baa6-b5076a6a5d73 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:40.947139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fad76f2-da1a-4788-baa6-b5076a6a5d73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:14:40.955234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-1fad76f2-da1a-4788-baa6-b5076a6a5d73 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 16:14:40.955234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:40.955234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:40.955234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:40.955234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:40.955234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:40.955234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:40.955234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:40.955234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:40.955234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:40.955234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:40.955234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:40.955234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:14:40.955234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:14:40.955234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:14:40.955234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:14:40.956406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:14:40.956406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:14:40.956406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:14:40.956406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:14:40.956406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:40.956406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:40.956406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:40.956406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:40.956406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:40.956406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:40.956406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:40.956406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:40.956406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:40.956406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:40.956406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:40.956406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:40.957255 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:40.957255 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:40.957255 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:40.957255 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:40.957255 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:14:40.957255 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:14:40.957255 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:14:40.957255 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:14:40.957255 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:14:40.957255 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:14:40.957255 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 16:14:40.957255 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:40.959545 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 519/1048] 10.222.0.67 () {56 vars in 1120 bytes} [Thu Jun 30 16:14:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:40.962998 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20111a4-e01e-405a-8df8-72b41328dbc3 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:40.974495 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01204ee7-c626-408a-aebd-d445aae8dda0 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:40.975109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01204ee7-c626-408a-aebd-d445aae8dda0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:40.975656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01204ee7-c626-408a-aebd-d445aae8dda0 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:40.977303 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 520/1049] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:14:40.982459 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:40.982607 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05128153-ccf6-445f-b40e-678a029afb21 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:40.982814 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05128153-ccf6-445f-b40e-678a029afb21 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:40.983072 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05128153-ccf6-445f-b40e-678a029afb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:41.005774 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-05128153-ccf6-445f-b40e-678a029afb21 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:41.005774 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-05128153-ccf6-445f-b40e-678a029afb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:41.077567 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 521/1050] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:40 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 16:14:41.087827 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20111a4-e01e-405a-8df8-72b41328dbc3 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:41.102939 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05128153-ccf6-445f-b40e-678a029afb21 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:41.115558 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d597feb-390f-4c85-8514-71ecea821718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:41.116225 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d597feb-390f-4c85-8514-71ecea821718 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:41.116503 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d597feb-390f-4c85-8514-71ecea821718 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:41.116774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d597feb-390f-4c85-8514-71ecea821718 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:41.117513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d597feb-390f-4c85-8514-71ecea821718 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:41.117815 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d597feb-390f-4c85-8514-71ecea821718 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:41.119185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d597feb-390f-4c85-8514-71ecea821718 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:41.127525 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 530/1051] 10.222.0.67 () {56 vars in 1115 bytes} [Thu Jun 30 16:14:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:41.133283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d597feb-390f-4c85-8514-71ecea821718 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:41.215150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05128153-ccf6-445f-b40e-678a029afb21 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:41.222805 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:41.226775 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:41.244839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d597feb-390f-4c85-8514-71ecea821718 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:41.257288 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c89e7-f77d-473c-960a-95486cbbc4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:41.258096 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c89e7-f77d-473c-960a-95486cbbc4cc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:41.258374 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c89e7-f77d-473c-960a-95486cbbc4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:41.258647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c89e7-f77d-473c-960a-95486cbbc4cc None admin] PATH_INFO: `/v3/services/image` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:41.259311 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c89e7-f77d-473c-960a-95486cbbc4cc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:41.260716 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30c89e7-f77d-473c-960a-95486cbbc4cc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:41.273886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e30c89e7-f77d-473c-960a-95486cbbc4cc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 16:14:41.273886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:41.273886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:14:41.273886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:14:41.273886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:14:41.273886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:14:41.273886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:14:41.273886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:41.273886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 16:14:41.273886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:41.273886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:14:41.273886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:41.273886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:41.273886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:41.273886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:41.273886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:41.273886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:41.275141 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:41.275141 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:41.275141 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:41.275141 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:41.275141 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:41.275141 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:41.275141 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:14:41.275141 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:14:41.275141 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:14:41.275141 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:14:41.275141 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:41.275141 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:41.275141 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:41.275141 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:41.275141 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:41.275141 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:41.276041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:41.276041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:41.276041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:41.276041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:41.276041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:41.276041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:41.276041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:41.276041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:41.276041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:41.276041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:41.276041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:14:41.276041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:14:41.276041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 16:14:41.276041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:41.276041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 522/1052] 10.222.0.67 () {56 vars in 1105 bytes} [Thu Jun 30 16:14:41 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:14:41.281175 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c89e7-f77d-473c-960a-95486cbbc4cc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:41.287609 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf653cf-cfff-4b9a-9d79-7245ed69e717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:41.294813 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf653cf-cfff-4b9a-9d79-7245ed69e717 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:41.294813 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf653cf-cfff-4b9a-9d79-7245ed69e717 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:41.294813 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf653cf-cfff-4b9a-9d79-7245ed69e717 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:41.300173 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf653cf-cfff-4b9a-9d79-7245ed69e717 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:41.301624 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf653cf-cfff-4b9a-9d79-7245ed69e717 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:41.318891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-9bf653cf-cfff-4b9a-9d79-7245ed69e717 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:41.318891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:41.318891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:41.318891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:41.318891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:41.318891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:41.318891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:41.318891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:41.318891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:41.318891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:41.318891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:41.318891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:41.318891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:41.318891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:41.318891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:41.320065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:41.320065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:41.320065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:41.320065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:41.320065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:41.320065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:41.320065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:41.320065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:41.320065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:41.320065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:41.320065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:41.320065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:41.320065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:41.320065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:41.320065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:41.320065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:41.320065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:41.321049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:41.321049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:41.321049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:41.321049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:41.323801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 531/1053] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:14:41.325692 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf653cf-cfff-4b9a-9d79-7245ed69e717 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:41.386859 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c89e7-f77d-473c-960a-95486cbbc4cc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:41.423727 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807eafb7-5d4d-4415-89c3-dae7c5836cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:41.425131 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807eafb7-5d4d-4415-89c3-dae7c5836cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:41.425432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807eafb7-5d4d-4415-89c3-dae7c5836cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:41.425710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807eafb7-5d4d-4415-89c3-dae7c5836cc6 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:41.426412 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807eafb7-5d4d-4415-89c3-dae7c5836cc6 None admin] RBAC: Adding query filter params (name=image) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:41.426712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807eafb7-5d4d-4415-89c3-dae7c5836cc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:41.428289 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807eafb7-5d4d-4415-89c3-dae7c5836cc6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:41.434160 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 523/1054] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:41.439680 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807eafb7-5d4d-4415-89c3-dae7c5836cc6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:41.445118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf653cf-cfff-4b9a-9d79-7245ed69e717 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:41.478509 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:41.478858 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:41.488755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b38a88-25bd-440a-b96b-ef1acb6647a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:41.489432 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b38a88-25bd-440a-b96b-ef1acb6647a4 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:41.489716 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b38a88-25bd-440a-b96b-ef1acb6647a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:41.489995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b38a88-25bd-440a-b96b-ef1acb6647a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:41.490722 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b38a88-25bd-440a-b96b-ef1acb6647a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:41.491035 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b38a88-25bd-440a-b96b-ef1acb6647a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:41.492465 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b38a88-25bd-440a-b96b-ef1acb6647a4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:41.505008 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 532/1055] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:41.506268 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e96666-3371-4eb8-bce1-5ccc7b1dc105 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:41.506560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e96666-3371-4eb8-bce1-5ccc7b1dc105 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:41.506841 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e96666-3371-4eb8-bce1-5ccc7b1dc105 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:41.508241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 533/1056] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:41.510805 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e96666-3371-4eb8-bce1-5ccc7b1dc105 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:41.546712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807eafb7-5d4d-4415-89c3-dae7c5836cc6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:41.582046 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473843c-a810-47e9-946e-2d241b014380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:41.582704 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473843c-a810-47e9-946e-2d241b014380 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:41.582992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473843c-a810-47e9-946e-2d241b014380 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:41.583269 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473843c-a810-47e9-946e-2d241b014380 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:41.584008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473843c-a810-47e9-946e-2d241b014380 None admin] RBAC: Adding query filter params (type=image) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:41.584312 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473843c-a810-47e9-946e-2d241b014380 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:41.585724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473843c-a810-47e9-946e-2d241b014380 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:41.590647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 524/1057] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:41.591863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba227a-fd2b-4993-88e8-7c33e59e6781 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:41.592149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba227a-fd2b-4993-88e8-7c33e59e6781 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:41.592429 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba227a-fd2b-4993-88e8-7c33e59e6781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:41.610424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-29ba227a-fd2b-4993-88e8-7c33e59e6781 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:41.612310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-29ba227a-fd2b-4993-88e8-7c33e59e6781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:41.617068 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e96666-3371-4eb8-bce1-5ccc7b1dc105 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:41.656908 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce737c1-82de-464b-ae9f-d4c1d4cc9f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:41.657693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce737c1-82de-464b-ae9f-d4c1d4cc9f3f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:41.657980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce737c1-82de-464b-ae9f-d4c1d4cc9f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:41.658253 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce737c1-82de-464b-ae9f-d4c1d4cc9f3f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:41.666396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce737c1-82de-464b-ae9f-d4c1d4cc9f3f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:41.667882 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce737c1-82de-464b-ae9f-d4c1d4cc9f3f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:41.680478 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 525/1058] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:41 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:41.681303 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba227a-fd2b-4993-88e8-7c33e59e6781 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:41.683799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-6ce737c1-82de-464b-ae9f-d4c1d4cc9f3f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 16:14:41.683799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:41.683799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:41.683799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:41.683799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:41.683799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:41.683799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:41.683799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:41.683799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:41.683799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:41.683799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:41.683799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:41.683799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:14:41.683799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:14:41.683799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:14:41.683799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:14:41.686142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:41.686142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:41.686142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:14:41.686142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:41.686142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:14:41.686142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:41.686142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:41.686142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:41.686142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:41.686142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:41.686142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:41.686142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:41.686142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:41.686142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:41.686142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:41.686142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:41.686142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:41.686142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:41.687345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:41.687345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:41.687345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:14:41.687345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:14:41.687345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:14:41.687345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:14:41.687345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:14:41.687345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:14:41.687345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 16:14:41.687345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:41.687345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 534/1059] 10.222.0.67 () {56 vars in 1093 bytes} [Thu Jun 30 16:14:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:14:41.690931 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce737c1-82de-464b-ae9f-d4c1d4cc9f3f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:41.717819 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:41.813477 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:41.864616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba227a-fd2b-4993-88e8-7c33e59e6781 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:41.865800 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce737c1-82de-464b-ae9f-d4c1d4cc9f3f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:41.900642 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd53d4c-2bed-4d16-9ac7-7443a5efdd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:41.900868 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79f472-8497-487f-b862-479fd122de4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:41.901301 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd53d4c-2bed-4d16-9ac7-7443a5efdd39 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:41.901449 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79f472-8497-487f-b862-479fd122de4f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:41.901576 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd53d4c-2bed-4d16-9ac7-7443a5efdd39 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:41.901675 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79f472-8497-487f-b862-479fd122de4f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:41.901846 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd53d4c-2bed-4d16-9ac7-7443a5efdd39 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:41.901938 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79f472-8497-487f-b862-479fd122de4f None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:41.902580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd53d4c-2bed-4d16-9ac7-7443a5efdd39 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:41.902671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd79f472-8497-487f-b862-479fd122de4f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:41.902986 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd79f472-8497-487f-b862-479fd122de4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:41.904017 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd53d4c-2bed-4d16-9ac7-7443a5efdd39 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:41.904489 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd79f472-8497-487f-b862-479fd122de4f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:41.922346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 535/1060] 10.222.0.67 () {56 vars in 1133 bytes} [Thu Jun 30 16:14:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:41.923176 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79f472-8497-487f-b862-479fd122de4f None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:41.987272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 526/1061] 10.222.0.67 () {60 vars in 1144 bytes} [Thu Jun 30 16:14:41 2022] POST /identity/v3/endpoints => generated 327 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:14:41.988278 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd53d4c-2bed-4d16-9ac7-7443a5efdd39 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:42.037078 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79f472-8497-487f-b862-479fd122de4f None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:42.048693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:42.086514 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743ad35-e0de-442f-95dc-fff815733b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:42.087351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743ad35-e0de-442f-95dc-fff815733b00 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:42.087668 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743ad35-e0de-442f-95dc-fff815733b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:42.088251 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743ad35-e0de-442f-95dc-fff815733b00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:42.100845 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743ad35-e0de-442f-95dc-fff815733b00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:14:42.105626 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:42.110334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-5743ad35-e0de-442f-95dc-fff815733b00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:14:42.110334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:42.110334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:42.110334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:42.110334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:42.110334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:42.110334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:42.110334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:42.110334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:42.110334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:42.110334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:42.110334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:42.110334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:14:42.110334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:14:42.110334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:14:42.110334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:14:42.111365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:14:42.111365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:14:42.111365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:14:42.111365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:14:42.111365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:42.111365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:42.111365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:42.111365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:42.111365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:42.111365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:42.111365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:42.111365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:42.111365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:42.111365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:42.111365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:42.111365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:42.112896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:42.112896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:42.112896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:42.112896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:42.112896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:14:42.112896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:14:42.112896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:14:42.112896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:14:42.112896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:14:42.112896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:14:42.112896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:14:42.112896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:42.112896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 536/1062] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:14:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:42.118365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743ad35-e0de-442f-95dc-fff815733b00 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:42.126687 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd53d4c-2bed-4d16-9ac7-7443a5efdd39 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:42.167993 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f969b-8a71-4f98-b985-d673b3337f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:42.168792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f969b-8a71-4f98-b985-d673b3337f58 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:42.169073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f969b-8a71-4f98-b985-d673b3337f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:42.169346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f969b-8a71-4f98-b985-d673b3337f58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:42.182200 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730f969b-8a71-4f98-b985-d673b3337f58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:42.183932 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730f969b-8a71-4f98-b985-d673b3337f58 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:42.203574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-730f969b-8a71-4f98-b985-d673b3337f58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:42.203574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:42.203574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:42.203574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:42.203574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:42.203574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:42.203574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:42.203574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:42.203574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:42.203574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:42.203574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:42.203574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:42.203574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:42.203574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:42.203574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:42.204554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:42.204554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:42.204554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:42.204554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:42.204554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:42.204554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:42.204554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:42.204554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:42.204554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:42.204554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:42.204554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:42.204554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:42.204554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:42.204554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:42.204554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:42.204554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:42.204554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:42.207346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:42.207346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:42.207346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:42.207346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:42.207346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 527/1063] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:42.210661 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f969b-8a71-4f98-b985-d673b3337f58 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:42.247711 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743ad35-e0de-442f-95dc-fff815733b00 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:42.290716 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f3b2e-cc86-43d3-b1c6-94e45184fcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:42.291668 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f3b2e-cc86-43d3-b1c6-94e45184fcc9 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:42.292120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f3b2e-cc86-43d3-b1c6-94e45184fcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:42.292638 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f3b2e-cc86-43d3-b1c6-94e45184fcc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:42.293723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f3b2e-cc86-43d3-b1c6-94e45184fcc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:42.294034 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f3b2e-cc86-43d3-b1c6-94e45184fcc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:42.295426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f3b2e-cc86-43d3-b1c6-94e45184fcc9 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:42.305677 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 537/1064] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:14:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:42.311325 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f3b2e-cc86-43d3-b1c6-94e45184fcc9 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:42.326355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f969b-8a71-4f98-b985-d673b3337f58 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:42.339710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:42.370091 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced26df6-2923-46b0-99ae-1f2a9d0f0d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:42.371156 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced26df6-2923-46b0-99ae-1f2a9d0f0d7f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:42.371709 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced26df6-2923-46b0-99ae-1f2a9d0f0d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:42.372169 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced26df6-2923-46b0-99ae-1f2a9d0f0d7f None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:42.373075 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced26df6-2923-46b0-99ae-1f2a9d0f0d7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:42.373669 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced26df6-2923-46b0-99ae-1f2a9d0f0d7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:42.375229 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced26df6-2923-46b0-99ae-1f2a9d0f0d7f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:42.382768 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:42.388975 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 528/1065] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:42.394645 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced26df6-2923-46b0-99ae-1f2a9d0f0d7f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:42.428545 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f3b2e-cc86-43d3-b1c6-94e45184fcc9 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:42.466315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a818a86-51f8-4cb9-a1e2-3359506bd210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:42.467189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a818a86-51f8-4cb9-a1e2-3359506bd210 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:42.467487 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a818a86-51f8-4cb9-a1e2-3359506bd210 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:42.467768 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a818a86-51f8-4cb9-a1e2-3359506bd210 None admin] PATH_INFO: `/v3/users/c753998fd15547fdbe31ce135dd957c0` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:42.485540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a818a86-51f8-4cb9-a1e2-3359506bd210 None admin] RBAC: Authorizing `identity:get_user(user_id=c753998fd15547fdbe31ce135dd957c0)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:42.488160 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a818a86-51f8-4cb9-a1e2-3359506bd210 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:42.494539 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 538/1066] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:14:42 2022] GET /identity/v3/users/c753998fd15547fdbe31ce135dd957c0 => generated 285 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:42.503279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a818a86-51f8-4cb9-a1e2-3359506bd210 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:42.512252 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced26df6-2923-46b0-99ae-1f2a9d0f0d7f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:42.551135 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65e71a-268a-4332-892d-1a62e5013596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:42.551957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65e71a-268a-4332-892d-1a62e5013596 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:42.552239 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65e71a-268a-4332-892d-1a62e5013596 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:42.552514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65e71a-268a-4332-892d-1a62e5013596 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:42.561167 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f65e71a-268a-4332-892d-1a62e5013596 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:42.562605 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f65e71a-268a-4332-892d-1a62e5013596 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:42.577234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-9f65e71a-268a-4332-892d-1a62e5013596 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:42.577234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:42.577234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:42.577234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:42.577234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:42.577234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:42.577234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:42.577234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:42.577234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:42.577234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:42.577234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:42.577234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:42.577234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:14:42.577234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:14:42.577234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:14:42.578363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:14:42.578363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:42.578363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:42.578363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:42.578363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:42.578363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:42.578363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:42.578363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:42.578363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:42.578363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:42.578363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:42.578363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:42.578363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:42.578363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:42.578363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:42.578363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:42.578363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:42.579518 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:14:42.579518 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:14:42.579518 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:14:42.579518 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:14:42.579518 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:14:42.579518 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:14:42.579518 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:42.579518 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:42.584075 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 529/1067] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:14:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:14:42.584932 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65e71a-268a-4332-892d-1a62e5013596 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:42.607500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:42.623697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a818a86-51f8-4cb9-a1e2-3359506bd210 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:42.650048 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:42.660929 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ed708-be08-452f-a237-e81f5d625896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:42.661683 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ed708-be08-452f-a237-e81f5d625896 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:42.661969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ed708-be08-452f-a237-e81f5d625896 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:42.662250 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ed708-be08-452f-a237-e81f5d625896 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:42.694127 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65e71a-268a-4332-892d-1a62e5013596 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:42.733853 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f116da4-f17c-4d80-af7e-f4c2c0bc5d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:42.735082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f116da4-f17c-4d80-af7e-f4c2c0bc5d3e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:42.735220 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ed708-be08-452f-a237-e81f5d625896 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:42.736007 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f116da4-f17c-4d80-af7e-f4c2c0bc5d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:42.736487 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f116da4-f17c-4d80-af7e-f4c2c0bc5d3e None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:42.736727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ed708-be08-452f-a237-e81f5d625896 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:42.737635 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f116da4-f17c-4d80-af7e-f4c2c0bc5d3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:42.738113 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f116da4-f17c-4d80-af7e-f4c2c0bc5d3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:42.740918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f116da4-f17c-4d80-af7e-f4c2c0bc5d3e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:42.757317 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 530/1068] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:14:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:42.763303 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f116da4-f17c-4d80-af7e-f4c2c0bc5d3e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:42.828009 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:42.845576 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 539/1069] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:14:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 16:14:42.851931 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ed708-be08-452f-a237-e81f5d625896 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:42.885524 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f116da4-f17c-4d80-af7e-f4c2c0bc5d3e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:42.905098 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:42.922122 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217260d9-6956-4690-9bf8-77df186be1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:42.923030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217260d9-6956-4690-9bf8-77df186be1e3 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:42.923306 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217260d9-6956-4690-9bf8-77df186be1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:42.923600 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217260d9-6956-4690-9bf8-77df186be1e3 None admin] PATH_INFO: `/v3/projects/19ef92ebcce94b90bf39ad3a58772a1d/users/71cca3708386459197abf6c67505bc5f/roles/c2206d1398e84c28b5365a335a04e5ac` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:42.926457 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217260d9-6956-4690-9bf8-77df186be1e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=19ef92ebcce94b90bf39ad3a58772a1d, user_id=71cca3708386459197abf6c67505bc5f, role_id=c2206d1398e84c28b5365a335a04e5ac)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:42.927920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217260d9-6956-4690-9bf8-77df186be1e3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:42.964356 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ed708-be08-452f-a237-e81f5d625896 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:43.000191 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbbeb9-1c7f-46ff-8be3-9e38c5f0eded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:43.004526 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbbeb9-1c7f-46ff-8be3-9e38c5f0eded None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:43.004526 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbbeb9-1c7f-46ff-8be3-9e38c5f0eded None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:43.004526 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbbeb9-1c7f-46ff-8be3-9e38c5f0eded None admin] PATH_INFO: `/v3/projects/26f5fb597b6943c7a1139be24fa10801` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:43.004526 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dbbeb9-1c7f-46ff-8be3-9e38c5f0eded None admin] RBAC: Authorizing `identity:get_project(project_id=26f5fb597b6943c7a1139be24fa10801)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:43.005186 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dbbeb9-1c7f-46ff-8be3-9e38c5f0eded None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:43.012371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 540/1070] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:14:42 2022] GET /identity/v3/projects/26f5fb597b6943c7a1139be24fa10801 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:43.013203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbbeb9-1c7f-46ff-8be3-9e38c5f0eded None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:43.027078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 531/1071] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 30 16:14:42 2022] PUT /identity/v3/projects/19ef92ebcce94b90bf39ad3a58772a1d/users/71cca3708386459197abf6c67505bc5f/roles/c2206d1398e84c28b5365a335a04e5ac => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:14:43.076528 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:43.133825 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbbeb9-1c7f-46ff-8be3-9e38c5f0eded None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:43.174977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebb32c-ab33-4292-af0e-f71dcae807c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:43.175633 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebb32c-ab33-4292-af0e-f71dcae807c2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:43.175985 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebb32c-ab33-4292-af0e-f71dcae807c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:43.176272 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebb32c-ab33-4292-af0e-f71dcae807c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:43.177147 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ebb32c-ab33-4292-af0e-f71dcae807c2 None admin] RBAC: Adding query filter params (role.id=8acc34a163874a60b03728b814eaf075, scope.project.id=26f5fb597b6943c7a1139be24fa10801, user.id=c753998fd15547fdbe31ce135dd957c0) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:43.177463 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ebb32c-ab33-4292-af0e-f71dcae807c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:43.179104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ebb32c-ab33-4292-af0e-f71dcae807c2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:43.186141 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 541/1072] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:14:43 2022] GET /identity/v3/role_assignments?user.id=c753998fd15547fdbe31ce135dd957c0&scope.project.id=26f5fb597b6943c7a1139be24fa10801&role.id=8acc34a163874a60b03728b814eaf075 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:44.098051 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec4893-3ac2-41b5-a6fe-f383e75f58f2 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:44.098562 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec4893-3ac2-41b5-a6fe-f383e75f58f2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:44.098640 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec4893-3ac2-41b5-a6fe-f383e75f58f2 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:44.100155 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 532/1073] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:44.105115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb1658-974c-4ff1-847e-a2b301167744 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:44.105579 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb1658-974c-4ff1-847e-a2b301167744 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:44.105661 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb1658-974c-4ff1-847e-a2b301167744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:44.122820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-abeb1658-974c-4ff1-847e-a2b301167744 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:44.124038 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-abeb1658-974c-4ff1-847e-a2b301167744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:44.198758 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 542/1074] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:44.222633 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec4893-3ac2-41b5-a6fe-f383e75f58f2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:44.339973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec4893-3ac2-41b5-a6fe-f383e75f58f2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:44.375692 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f4365-bc61-4c9a-af42-c586935580c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:44.376345 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f4365-bc61-4c9a-af42-c586935580c2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:44.376627 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f4365-bc61-4c9a-af42-c586935580c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:44.377426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f4365-bc61-4c9a-af42-c586935580c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:44.424805 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:44.447896 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f4365-bc61-4c9a-af42-c586935580c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:44.449353 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f4365-bc61-4c9a-af42-c586935580c2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:44.549742 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 533/1075] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:14:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 16:14:44.555487 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb1658-974c-4ff1-847e-a2b301167744 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:44.603442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:44.657973 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb1658-974c-4ff1-847e-a2b301167744 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:44.692631 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4cb57c-d377-41ec-9020-dfe34deba586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:44.693425 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cb57c-d377-41ec-9020-dfe34deba586 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:44.693713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cb57c-d377-41ec-9020-dfe34deba586 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:44.693993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4cb57c-d377-41ec-9020-dfe34deba586 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:44.704896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4cb57c-d377-41ec-9020-dfe34deba586 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:44.706921 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4cb57c-d377-41ec-9020-dfe34deba586 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:44.724357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-db4cb57c-d377-41ec-9020-dfe34deba586 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:44.724357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:44.724357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:44.724357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:44.724357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:44.724357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:44.724357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:44.724357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:44.724357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:44.724357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:44.724357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:44.724357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:44.724357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:44.724357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:44.724357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:44.725365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:44.725365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:44.725365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:44.725365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:44.725365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:44.725365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:44.725365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:44.725365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:44.725365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:44.725365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:44.725365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:44.725365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:44.725365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:44.725365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:44.725365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:44.725365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:44.725365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:44.726348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:44.726348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:44.726348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:44.726348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:44.727565 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 543/1076] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:44.731537 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f4365-bc61-4c9a-af42-c586935580c2 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:44.813307 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:44.835115 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f4365-bc61-4c9a-af42-c586935580c2 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:44.870666 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfed236-162c-47a8-b139-a81baed26c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:44.871248 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfed236-162c-47a8-b139-a81baed26c5c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:44.871568 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfed236-162c-47a8-b139-a81baed26c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:44.871836 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfed236-162c-47a8-b139-a81baed26c5c None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:44.872552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfed236-162c-47a8-b139-a81baed26c5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:44.872863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfed236-162c-47a8-b139-a81baed26c5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:44.874231 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfed236-162c-47a8-b139-a81baed26c5c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:44.891634 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 534/1077] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:45.102470 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14392702-bf2a-43af-b703-67b320375c40 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:45.102852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14392702-bf2a-43af-b703-67b320375c40 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:45.103055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14392702-bf2a-43af-b703-67b320375c40 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:45.104459 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 544/1078] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:45.109675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e6c32-fec0-4886-8436-38c55068fe67 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:45.110126 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e6c32-fec0-4886-8436-38c55068fe67 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:45.110221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e6c32-fec0-4886-8436-38c55068fe67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:45.128404 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-422e6c32-fec0-4886-8436-38c55068fe67 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:45.129532 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-422e6c32-fec0-4886-8436-38c55068fe67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:45.199282 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 535/1079] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:45.223331 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14392702-bf2a-43af-b703-67b320375c40 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:45.281156 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6031950-95e4-4ff9-a335-6656d5c8b0a5 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:45.281463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6031950-95e4-4ff9-a335-6656d5c8b0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:45.281745 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6031950-95e4-4ff9-a335-6656d5c8b0a5 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:45.283160 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 536/1080] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:14:45.288365 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e035382-22b5-417e-8946-507dc5e6596e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:45.288652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e035382-22b5-417e-8946-507dc5e6596e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:45.288930 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e035382-22b5-417e-8946-507dc5e6596e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:45.305797 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2e035382-22b5-417e-8946-507dc5e6596e None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:45.307012 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2e035382-22b5-417e-8946-507dc5e6596e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:45.323635 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:45.328262 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14392702-bf2a-43af-b703-67b320375c40 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:45.363112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b36819-5967-4f33-bfc5-b7679e6a3e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:45.363945 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b36819-5967-4f33-bfc5-b7679e6a3e65 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:45.364230 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b36819-5967-4f33-bfc5-b7679e6a3e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:45.364509 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b36819-5967-4f33-bfc5-b7679e6a3e65 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:45.370814 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:45.375084 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b36819-5967-4f33-bfc5-b7679e6a3e65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:14:45.376428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 537/1081] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 16:14:45.384303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f8b36819-5967-4f33-bfc5-b7679e6a3e65 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 16:14:45.384303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:45.384303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:45.384303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:45.384303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:45.384303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:45.384303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:45.384303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:45.384303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:45.384303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:45.384303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:45.384303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:45.384303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:14:45.384303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:14:45.384303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:14:45.384303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:14:45.385270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:14:45.385270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:14:45.385270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:14:45.385270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:14:45.385270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:45.385270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:45.385270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:45.385270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:45.385270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:45.385270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:45.385270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:45.385270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:45.385270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:45.385270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:45.385270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:45.385270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:45.386112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:45.386112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:45.386112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:45.386112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:45.386112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:14:45.386112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:14:45.386112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:14:45.386112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:14:45.386112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:14:45.386112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:14:45.386112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 16:14:45.386112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:45.387617 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 545/1082] 10.222.0.67 () {56 vars in 1120 bytes} [Thu Jun 30 16:14:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:45.391107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e035382-22b5-417e-8946-507dc5e6596e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:45.400272 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b36819-5967-4f33-bfc5-b7679e6a3e65 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:45.492923 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e035382-22b5-417e-8946-507dc5e6596e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:45.501533 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b36819-5967-4f33-bfc5-b7679e6a3e65 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:45.527412 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8b3fc-6367-4380-a4cb-2d590fe4fea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:45.528102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8b3fc-6367-4380-a4cb-2d590fe4fea6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:45.528386 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8b3fc-6367-4380-a4cb-2d590fe4fea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:45.528670 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac8b3fc-6367-4380-a4cb-2d590fe4fea6 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:45.529410 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8b3fc-6367-4380-a4cb-2d590fe4fea6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:45.529722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8b3fc-6367-4380-a4cb-2d590fe4fea6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:45.531112 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac8b3fc-6367-4380-a4cb-2d590fe4fea6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:45.536353 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c81a0-3243-490d-bf2a-66065fb6d932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:45.537152 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c81a0-3243-490d-bf2a-66065fb6d932 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:45.537434 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c81a0-3243-490d-bf2a-66065fb6d932 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:45.537711 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c81a0-3243-490d-bf2a-66065fb6d932 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:45.539527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 538/1083] 10.222.0.67 () {56 vars in 1115 bytes} [Thu Jun 30 16:14:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:45.545173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8b3fc-6367-4380-a4cb-2d590fe4fea6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:45.548428 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c81a0-3243-490d-bf2a-66065fb6d932 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:14:45.556919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-270c81a0-3243-490d-bf2a-66065fb6d932 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:14:45.556919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:45.556919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:45.556919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:45.556919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:45.556919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:45.556919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:45.556919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:45.556919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:45.556919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:45.556919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:45.556919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:45.556919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:14:45.556919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:14:45.556919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:14:45.556919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:14:45.557906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:14:45.557906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:14:45.557906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:14:45.557906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:14:45.557906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:45.557906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:45.557906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:45.557906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:45.557906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:45.557906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:45.557906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:45.557906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:45.557906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:45.557906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:45.557906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:45.557906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:45.558748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:45.558748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:45.558748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:45.558748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:45.558748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:14:45.558748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:14:45.558748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:14:45.558748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:14:45.558748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:14:45.558748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:14:45.558748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:14:45.558748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:45.558748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 546/1084] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:14:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:14:45.564233 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c81a0-3243-490d-bf2a-66065fb6d932 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:45.584222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:45.622270 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:45.662754 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac8b3fc-6367-4380-a4cb-2d590fe4fea6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:45.665358 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c81a0-3243-490d-bf2a-66065fb6d932 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:45.696734 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a2dc0-34ce-4e7b-9260-d4eecc7cbd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:45.697532 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a2dc0-34ce-4e7b-9260-d4eecc7cbd8e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:45.697813 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a2dc0-34ce-4e7b-9260-d4eecc7cbd8e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:45.698084 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a2dc0-34ce-4e7b-9260-d4eecc7cbd8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:45.699919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1318d-355a-4172-8c55-db5cd7066085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:45.700578 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1318d-355a-4172-8c55-db5cd7066085 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:45.700860 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1318d-355a-4172-8c55-db5cd7066085 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:45.701134 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1318d-355a-4172-8c55-db5cd7066085 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:45.701870 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1318d-355a-4172-8c55-db5cd7066085 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:45.702178 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1318d-355a-4172-8c55-db5cd7066085 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:45.703572 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1318d-355a-4172-8c55-db5cd7066085 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:45.708703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a2dc0-34ce-4e7b-9260-d4eecc7cbd8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:45.710119 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99a2dc0-34ce-4e7b-9260-d4eecc7cbd8e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:45.712180 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 547/1085] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:14:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:45.717816 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1318d-355a-4172-8c55-db5cd7066085 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:45.727648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d99a2dc0-34ce-4e7b-9260-d4eecc7cbd8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:45.727648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:45.727648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:45.727648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:45.727648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:45.727648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:45.727648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:45.727648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:45.727648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:45.727648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:45.727648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:45.727648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:45.727648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:45.727648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:45.727648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:45.729109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:45.729109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:45.729109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:45.729109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:45.729109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:45.729109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:45.729109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:45.729109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:45.729109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:45.729109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:45.729109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:45.729109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:45.729109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:45.729109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:45.729109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:45.729109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:45.729109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:45.730671 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:45.730671 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:45.730671 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:45.730671 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:45.730671 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 539/1086] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:14:45.734832 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a2dc0-34ce-4e7b-9260-d4eecc7cbd8e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:45.804191 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:45.823973 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1318d-355a-4172-8c55-db5cd7066085 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:45.837620 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a2dc0-34ce-4e7b-9260-d4eecc7cbd8e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:45.850478 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:45.858130 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e20a1-aa93-4ca7-bc70-930cce227b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:45.859025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e20a1-aa93-4ca7-bc70-930cce227b06 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:45.859274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e20a1-aa93-4ca7-bc70-930cce227b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:45.859581 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e20a1-aa93-4ca7-bc70-930cce227b06 None admin] PATH_INFO: `/v3/users/c753998fd15547fdbe31ce135dd957c0` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:45.860770 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e20a1-aa93-4ca7-bc70-930cce227b06 None admin] RBAC: Authorizing `identity:get_user(user_id=c753998fd15547fdbe31ce135dd957c0)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:45.862780 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32e20a1-aa93-4ca7-bc70-930cce227b06 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:45.864445 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 548/1087] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:14:45 2022] GET /identity/v3/users/c753998fd15547fdbe31ce135dd957c0 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:45.870347 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e20a1-aa93-4ca7-bc70-930cce227b06 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:45.871883 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b014516-94d8-4021-856e-c4340e1ba117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:45.872540 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b014516-94d8-4021-856e-c4340e1ba117 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:45.872826 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b014516-94d8-4021-856e-c4340e1ba117 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:45.873356 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b014516-94d8-4021-856e-c4340e1ba117 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:45.873818 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b014516-94d8-4021-856e-c4340e1ba117 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:45.874121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b014516-94d8-4021-856e-c4340e1ba117 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:45.875496 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b014516-94d8-4021-856e-c4340e1ba117 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:45.891509 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 540/1088] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:45.895295 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b014516-94d8-4021-856e-c4340e1ba117 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:45.971696 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e20a1-aa93-4ca7-bc70-930cce227b06 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:45.999625 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b014516-94d8-4021-856e-c4340e1ba117 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:46.010825 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e6aa8-e2c6-4939-a6f0-ca8e405584dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:46.011714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e6aa8-e2c6-4939-a6f0-ca8e405584dd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:46.011998 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e6aa8-e2c6-4939-a6f0-ca8e405584dd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:46.012272 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e6aa8-e2c6-4939-a6f0-ca8e405584dd None admin] PATH_INFO: `/v3/projects/26f5fb597b6943c7a1139be24fa10801` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:46.013450 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e6aa8-e2c6-4939-a6f0-ca8e405584dd None admin] RBAC: Authorizing `identity:get_project(project_id=26f5fb597b6943c7a1139be24fa10801)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:46.014897 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e6aa8-e2c6-4939-a6f0-ca8e405584dd None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:46.016725 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 549/1089] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:14:45 2022] GET /identity/v3/projects/26f5fb597b6943c7a1139be24fa10801 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:46.022979 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e6aa8-e2c6-4939-a6f0-ca8e405584dd None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:46.036080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96225c58-e2a4-4f91-aa99-d94db734c4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:46.036867 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96225c58-e2a4-4f91-aa99-d94db734c4fc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:46.037139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96225c58-e2a4-4f91-aa99-d94db734c4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:46.037404 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96225c58-e2a4-4f91-aa99-d94db734c4fc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:46.041166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:46.044472 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96225c58-e2a4-4f91-aa99-d94db734c4fc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:46.045921 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96225c58-e2a4-4f91-aa99-d94db734c4fc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:46.061460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-96225c58-e2a4-4f91-aa99-d94db734c4fc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 16:14:46.061460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:46.061460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:46.061460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:46.061460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:46.061460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:46.061460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:46.061460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:46.061460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:46.061460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:46.061460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:46.061460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:46.061460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:14:46.061460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:14:46.061460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:14:46.061460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:14:46.062453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:46.062453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:46.062453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:14:46.062453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:46.062453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:14:46.062453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:14:46.062453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:46.062453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:46.062453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:46.062453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:46.062453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:46.062453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:46.062453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:46.062453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:46.062453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:46.062453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:46.062453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:46.062453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:46.063654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:46.063654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:46.063654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:14:46.063654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:14:46.063654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:14:46.063654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:14:46.063654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:14:46.063654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:14:46.063654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 16:14:46.063654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:46.063654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 541/1090] 10.222.0.67 () {56 vars in 1093 bytes} [Thu Jun 30 16:14:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 16:14:46.068383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96225c58-e2a4-4f91-aa99-d94db734c4fc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:46.099777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:46.124038 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e6aa8-e2c6-4939-a6f0-ca8e405584dd None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:46.158836 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a02d39-7797-4bc2-9586-e5903d0adbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:46.159703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a02d39-7797-4bc2-9586-e5903d0adbb3 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:46.159989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a02d39-7797-4bc2-9586-e5903d0adbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:46.160267 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a02d39-7797-4bc2-9586-e5903d0adbb3 None admin] PATH_INFO: `/v3/projects/26f5fb597b6943c7a1139be24fa10801/users/c753998fd15547fdbe31ce135dd957c0/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:46.162294 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a02d39-7797-4bc2-9586-e5903d0adbb3 None admin] RBAC: Authorizing `identity:create_grant(project_id=26f5fb597b6943c7a1139be24fa10801, user_id=c753998fd15547fdbe31ce135dd957c0, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:46.163771 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a02d39-7797-4bc2-9586-e5903d0adbb3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:46.169642 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96225c58-e2a4-4f91-aa99-d94db734c4fc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:46.204157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc1702-706f-4450-92a0-930656df5e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:46.204820 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc1702-706f-4450-92a0-930656df5e5b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:46.205315 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc1702-706f-4450-92a0-930656df5e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:46.205315 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc1702-706f-4450-92a0-930656df5e5b None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:46.206215 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc1702-706f-4450-92a0-930656df5e5b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:46.206525 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc1702-706f-4450-92a0-930656df5e5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:46.207992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc1702-706f-4450-92a0-930656df5e5b None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:46.225414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 542/1091] 10.222.0.67 () {56 vars in 1133 bytes} [Thu Jun 30 16:14:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:14:46.230955 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc1702-706f-4450-92a0-930656df5e5b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:46.243951 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 550/1092] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 30 16:14:46 2022] PUT /identity/v3/projects/26f5fb597b6943c7a1139be24fa10801/users/c753998fd15547fdbe31ce135dd957c0/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:14:46.332646 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:46.342615 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc1702-706f-4450-92a0-930656df5e5b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:46.380722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19724228-1824-48f0-be07-eda8236d4143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:46.381386 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19724228-1824-48f0-be07-eda8236d4143 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:46.381669 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19724228-1824-48f0-be07-eda8236d4143 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:46.381946 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19724228-1824-48f0-be07-eda8236d4143 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:46.466432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19724228-1824-48f0-be07-eda8236d4143 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:46.467909 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19724228-1824-48f0-be07-eda8236d4143 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:46.558982 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:46.569254 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 543/1093] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:14:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 16:14:46.575017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a02d39-7797-4bc2-9586-e5903d0adbb3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:46.654002 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:46.676069 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a02d39-7797-4bc2-9586-e5903d0adbb3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:46.711670 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc424d16-ed99-483f-b868-148ce04e9e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:46.712572 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc424d16-ed99-483f-b868-148ce04e9e64 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:46.713268 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc424d16-ed99-483f-b868-148ce04e9e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:46.713542 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc424d16-ed99-483f-b868-148ce04e9e64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:46.724904 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc424d16-ed99-483f-b868-148ce04e9e64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:46.726352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc424d16-ed99-483f-b868-148ce04e9e64 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:46.743869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-cc424d16-ed99-483f-b868-148ce04e9e64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:46.743869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:46.743869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:46.743869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:46.743869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:46.743869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:46.743869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:46.743869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:46.743869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:46.743869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:46.743869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:46.743869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:46.743869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:14:46.743869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:14:46.743869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:14:46.744918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:14:46.744918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:46.744918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:46.744918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:46.744918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:46.744918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:46.744918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:46.744918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:46.744918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:46.744918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:46.744918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:46.744918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:46.744918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:46.744918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:46.744918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:46.744918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:46.744918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:46.745807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:14:46.745807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:14:46.745807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:14:46.745807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:46.745807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 551/1094] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:46.750812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19724228-1824-48f0-be07-eda8236d4143 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:46.850375 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19724228-1824-48f0-be07-eda8236d4143 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:46.886438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dff631-cdbe-4f68-bb7f-a25bd4b71aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:46.887145 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dff631-cdbe-4f68-bb7f-a25bd4b71aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:46.887424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dff631-cdbe-4f68-bb7f-a25bd4b71aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:46.887727 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dff631-cdbe-4f68-bb7f-a25bd4b71aa0 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:46.888437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dff631-cdbe-4f68-bb7f-a25bd4b71aa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:46.888743 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dff631-cdbe-4f68-bb7f-a25bd4b71aa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:46.890113 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dff631-cdbe-4f68-bb7f-a25bd4b71aa0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:46.911187 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 544/1095] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:46.912545 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc424d16-ed99-483f-b868-148ce04e9e64 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:47.017501 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc424d16-ed99-483f-b868-148ce04e9e64 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:47.044946 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:47.053610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e5e5f-896e-423c-aa79-59877015f1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:47.058539 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e5e5f-896e-423c-aa79-59877015f1e3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:47.058539 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e5e5f-896e-423c-aa79-59877015f1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:47.058539 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e5e5f-896e-423c-aa79-59877015f1e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:47.061480 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e5e5f-896e-423c-aa79-59877015f1e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:47.062906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e5e5f-896e-423c-aa79-59877015f1e3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:47.078053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-900e5e5f-896e-423c-aa79-59877015f1e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:47.078053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:47.078053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:47.078053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:47.078053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:47.078053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:47.078053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:47.078053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:47.078053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:47.078053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:47.078053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:47.078053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:47.078053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:14:47.078053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:14:47.078053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:14:47.078993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:14:47.078993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:47.078993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:47.078993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:47.078993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:47.078993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:47.078993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:47.078993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:47.078993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:47.078993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:47.078993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:47.078993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:47.078993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:47.078993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:47.078993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:47.078993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:47.078993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:47.079943 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:14:47.079943 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:14:47.079943 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:14:47.079943 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:14:47.079943 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:14:47.079943 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:14:47.079943 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:14:47.079943 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:14:47.079943 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 552/1096] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:14:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:47.085772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dff631-cdbe-4f68-bb7f-a25bd4b71aa0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:47.142334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:47.195729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dff631-cdbe-4f68-bb7f-a25bd4b71aa0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:47.238699 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c670265-0500-4db8-ac13-c473d00ffcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:47.239362 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c670265-0500-4db8-ac13-c473d00ffcd0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:47.239673 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c670265-0500-4db8-ac13-c473d00ffcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:47.239946 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c670265-0500-4db8-ac13-c473d00ffcd0 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:47.240725 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c670265-0500-4db8-ac13-c473d00ffcd0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:47.241033 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c670265-0500-4db8-ac13-c473d00ffcd0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:47.242462 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c670265-0500-4db8-ac13-c473d00ffcd0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:47.255864 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 545/1097] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:14:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:47.261819 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e5e5f-896e-423c-aa79-59877015f1e3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:47.361617 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e5e5f-896e-423c-aa79-59877015f1e3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:47.399894 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf6d43-f92d-4db8-b75d-30777c04a17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:47.400474 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf6d43-f92d-4db8-b75d-30777c04a17a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:47.400754 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf6d43-f92d-4db8-b75d-30777c04a17a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:47.401030 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf6d43-f92d-4db8-b75d-30777c04a17a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:47.401860 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf6d43-f92d-4db8-b75d-30777c04a17a None admin] RBAC: Adding query filter params (role.id=c2206d1398e84c28b5365a335a04e5ac, scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d, user.id=71cca3708386459197abf6c67505bc5f) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:47.402171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf6d43-f92d-4db8-b75d-30777c04a17a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:47.403637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf6d43-f92d-4db8-b75d-30777c04a17a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:47.410594 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 553/1098] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:14:47 2022] GET /identity/v3/role_assignments?user.id=71cca3708386459197abf6c67505bc5f&scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d&role.id=c2206d1398e84c28b5365a335a04e5ac => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:48.318905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57b80d-e033-4fbb-b435-118a10c0fa08 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:48.319517 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57b80d-e033-4fbb-b435-118a10c0fa08 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:48.319517 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57b80d-e033-4fbb-b435-118a10c0fa08 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:48.320872 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 546/1099] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:48.326032 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c29cc-67b2-4936-8a44-617869463055 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:48.326459 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c29cc-67b2-4936-8a44-617869463055 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:48.326642 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c29cc-67b2-4936-8a44-617869463055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:48.343582 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-691c29cc-67b2-4936-8a44-617869463055 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:48.344757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-691c29cc-67b2-4936-8a44-617869463055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:48.393624 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:48.413715 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 554/1100] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:48.438571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57b80d-e033-4fbb-b435-118a10c0fa08 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:48.550070 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57b80d-e033-4fbb-b435-118a10c0fa08 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:48.560547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:48.583462 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b0e71-4704-436b-bdf8-7cb7e83bc9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:48.584272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b0e71-4704-436b-bdf8-7cb7e83bc9fd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:48.584574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b0e71-4704-436b-bdf8-7cb7e83bc9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:48.585052 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507b0e71-4704-436b-bdf8-7cb7e83bc9fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:48.595242 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507b0e71-4704-436b-bdf8-7cb7e83bc9fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:14:48.603552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-507b0e71-4704-436b-bdf8-7cb7e83bc9fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:14:48.603552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:14:48.603552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:14:48.603552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:14:48.603552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:14:48.603552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:14:48.603552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:14:48.603552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:14:48.603552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:14:48.603552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:14:48.603552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:14:48.603552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:14:48.603552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:14:48.603552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:14:48.603552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:14:48.603552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:14:48.604652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:14:48.604652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:14:48.604652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:14:48.604652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:14:48.604652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:14:48.604652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:14:48.604652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:14:48.604652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:14:48.604652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:14:48.604652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:14:48.604652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:14:48.604652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:14:48.604652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:14:48.604652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:14:48.604652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:14:48.604652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:14:48.605698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:14:48.605698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:14:48.605698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:14:48.605698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:14:48.605698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:14:48.605698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:14:48.605698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:14:48.605698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:14:48.605698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:14:48.605698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:14:48.605698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:14:48.605698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:14:48.605698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 547/1101] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:14:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:14:48.610853 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c29cc-67b2-4936-8a44-617869463055 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:48.707575 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c29cc-67b2-4936-8a44-617869463055 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:48.740664 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84867063-4691-40e6-80d3-5ab6ec0d1338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:48.741329 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84867063-4691-40e6-80d3-5ab6ec0d1338 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:48.741613 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84867063-4691-40e6-80d3-5ab6ec0d1338 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:48.741900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84867063-4691-40e6-80d3-5ab6ec0d1338 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:48.742891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84867063-4691-40e6-80d3-5ab6ec0d1338 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:48.743198 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84867063-4691-40e6-80d3-5ab6ec0d1338 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:48.744636 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84867063-4691-40e6-80d3-5ab6ec0d1338 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:48.752983 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 555/1102] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:14:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:48.758780 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b0e71-4704-436b-bdf8-7cb7e83bc9fd None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:48.859202 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507b0e71-4704-436b-bdf8-7cb7e83bc9fd None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:48.894537 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35fbb8-9c89-4551-9488-6bb6b6191631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:48.895396 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35fbb8-9c89-4551-9488-6bb6b6191631 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:48.895934 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35fbb8-9c89-4551-9488-6bb6b6191631 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:48.896210 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f35fbb8-9c89-4551-9488-6bb6b6191631 None admin] PATH_INFO: `/v3/users/c753998fd15547fdbe31ce135dd957c0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:48.897686 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35fbb8-9c89-4551-9488-6bb6b6191631 None admin] RBAC: Authorizing `identity:get_user(user_id=c753998fd15547fdbe31ce135dd957c0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:48.899189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f35fbb8-9c89-4551-9488-6bb6b6191631 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:48.900891 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 548/1103] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:14:48 2022] GET /identity/v3/users/c753998fd15547fdbe31ce135dd957c0 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:48.906669 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84867063-4691-40e6-80d3-5ab6ec0d1338 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:48.952303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:49.006070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84867063-4691-40e6-80d3-5ab6ec0d1338 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:49.039586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b439f0-ba1a-40fd-a673-d6fbc01d7402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:49.040234 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b439f0-ba1a-40fd-a673-d6fbc01d7402 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:49.040518 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b439f0-ba1a-40fd-a673-d6fbc01d7402 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:49.040792 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b439f0-ba1a-40fd-a673-d6fbc01d7402 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:49.107134 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b439f0-ba1a-40fd-a673-d6fbc01d7402 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:49.108617 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b439f0-ba1a-40fd-a673-d6fbc01d7402 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:49.152798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:49.213323 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 556/1104] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:14:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 16:14:49.219310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35fbb8-9c89-4551-9488-6bb6b6191631 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:49.258905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:49.318636 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f35fbb8-9c89-4551-9488-6bb6b6191631 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:49.350844 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d00c1-b8c1-4652-8419-755cd0dfb20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:49.351718 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d00c1-b8c1-4652-8419-755cd0dfb20e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:49.351997 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d00c1-b8c1-4652-8419-755cd0dfb20e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:49.352268 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d00c1-b8c1-4652-8419-755cd0dfb20e None admin] PATH_INFO: `/v3/projects/26f5fb597b6943c7a1139be24fa10801` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:49.353428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d00c1-b8c1-4652-8419-755cd0dfb20e None admin] RBAC: Authorizing `identity:get_project(project_id=26f5fb597b6943c7a1139be24fa10801)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:49.354841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d00c1-b8c1-4652-8419-755cd0dfb20e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:49.356589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 549/1105] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:14:49 2022] GET /identity/v3/projects/26f5fb597b6943c7a1139be24fa10801 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:49.363209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b439f0-ba1a-40fd-a673-d6fbc01d7402 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:49.461866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b439f0-ba1a-40fd-a673-d6fbc01d7402 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:49.496134 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f725c5-31d8-46ee-9e6b-68fe4d607e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:49.496784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f725c5-31d8-46ee-9e6b-68fe4d607e13 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:49.497068 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f725c5-31d8-46ee-9e6b-68fe4d607e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:49.497346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f725c5-31d8-46ee-9e6b-68fe4d607e13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:49.498277 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f725c5-31d8-46ee-9e6b-68fe4d607e13 None admin] RBAC: Adding query filter params (role.id=8acc34a163874a60b03728b814eaf075, scope.project.id=26f5fb597b6943c7a1139be24fa10801, user.id=c753998fd15547fdbe31ce135dd957c0) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:49.498599 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f725c5-31d8-46ee-9e6b-68fe4d607e13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:49.500083 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f725c5-31d8-46ee-9e6b-68fe4d607e13 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:49.506418 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 557/1106] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:14:49 2022] GET /identity/v3/role_assignments?user.id=c753998fd15547fdbe31ce135dd957c0&scope.project.id=26f5fb597b6943c7a1139be24fa10801&role.id=8acc34a163874a60b03728b814eaf075 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:51.580775 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552c5a2-3b55-4626-aa08-41d051919ebf None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:51.581589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552c5a2-3b55-4626-aa08-41d051919ebf None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:51.581589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552c5a2-3b55-4626-aa08-41d051919ebf None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:51.582962 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 550/1107] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:51.588021 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2b7ca-b7d8-4929-b53e-11c6b8d0ad80 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:51.588423 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2b7ca-b7d8-4929-b53e-11c6b8d0ad80 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:51.588582 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca2b7ca-b7d8-4929-b53e-11c6b8d0ad80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:51.605279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1ca2b7ca-b7d8-4929-b53e-11c6b8d0ad80 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:51.606469 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1ca2b7ca-b7d8-4929-b53e-11c6b8d0ad80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:51.608480 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:51.674308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 558/1108] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 16:14:51.699941 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552c5a2-3b55-4626-aa08-41d051919ebf None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:51.801420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552c5a2-3b55-4626-aa08-41d051919ebf None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:51.808695 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:51.834955 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4c8e1d-5190-47a9-ab82-3defda9146bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:51.835739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c8e1d-5190-47a9-ab82-3defda9146bf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:51.836023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c8e1d-5190-47a9-ab82-3defda9146bf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:51.836301 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4c8e1d-5190-47a9-ab82-3defda9146bf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:51.837477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4c8e1d-5190-47a9-ab82-3defda9146bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:51.838909 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4c8e1d-5190-47a9-ab82-3defda9146bf None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:51.840644 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 551/1109] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:14:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:51.847170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca2b7ca-b7d8-4929-b53e-11c6b8d0ad80 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:51.946655 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca2b7ca-b7d8-4929-b53e-11c6b8d0ad80 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:51.980457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a53e6f-aecc-4d65-b737-c313a50719bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:51.981114 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a53e6f-aecc-4d65-b737-c313a50719bb None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:51.981393 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a53e6f-aecc-4d65-b737-c313a50719bb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:51.981667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a53e6f-aecc-4d65-b737-c313a50719bb None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:51.982361 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a53e6f-aecc-4d65-b737-c313a50719bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:51.983813 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a53e6f-aecc-4d65-b737-c313a50719bb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:51.993302 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:52.156110 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 559/1110] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 30 16:14:51 2022] POST /identity/v3/users => generated 286 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:14:54.245033 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdf900-60c2-499d-8ca7-ac4a1d5f1bbb None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:54.245590 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdf900-60c2-499d-8ca7-ac4a1d5f1bbb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:54.245590 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdf900-60c2-499d-8ca7-ac4a1d5f1bbb None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:54.246951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 552/1111] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:54.252713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7eeb65-a024-4dbe-833b-fd62f88290a6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:54.253126 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7eeb65-a024-4dbe-833b-fd62f88290a6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:54.253482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7eeb65-a024-4dbe-833b-fd62f88290a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:54.270807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8b7eeb65-a024-4dbe-833b-fd62f88290a6 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:54.272425 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8b7eeb65-a024-4dbe-833b-fd62f88290a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:54.338476 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 560/1112] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:54.362090 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbdf900-60c2-499d-8ca7-ac4a1d5f1bbb None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:54.463108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbdf900-60c2-499d-8ca7-ac4a1d5f1bbb None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:54.497511 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed32f51-6a21-47ff-b2e0-7a30ba83a7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:54.498383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed32f51-6a21-47ff-b2e0-7a30ba83a7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:54.498663 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed32f51-6a21-47ff-b2e0-7a30ba83a7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:54.498936 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed32f51-6a21-47ff-b2e0-7a30ba83a7e0 None admin] PATH_INFO: `/v3/roles/cfeec012f4ec48138c0e637e9a7fde61` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:54.500429 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed32f51-6a21-47ff-b2e0-7a30ba83a7e0 None admin] RBAC: Authorizing `identity:get_role(role_id=cfeec012f4ec48138c0e637e9a7fde61)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:54.501808 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed32f51-6a21-47ff-b2e0-7a30ba83a7e0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:54.503888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 553/1113] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:14:54 2022] GET /identity/v3/roles/cfeec012f4ec48138c0e637e9a7fde61 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:54.509608 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7eeb65-a024-4dbe-833b-fd62f88290a6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:54.607404 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7eeb65-a024-4dbe-833b-fd62f88290a6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:54.631492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:54.641575 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd5c88-d039-4b90-adf6-f5bf1bcbef06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:54.645715 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd5c88-d039-4b90-adf6-f5bf1bcbef06 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:54.645715 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd5c88-d039-4b90-adf6-f5bf1bcbef06 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:54.645715 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cd5c88-d039-4b90-adf6-f5bf1bcbef06 None admin] PATH_INFO: `/v3/users/89964c517aa44ec1a4f2f9b0694cdc82` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:54.656218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd5c88-d039-4b90-adf6-f5bf1bcbef06 None admin] RBAC: Authorizing `identity:get_user(user_id=89964c517aa44ec1a4f2f9b0694cdc82)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:54.657654 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cd5c88-d039-4b90-adf6-f5bf1bcbef06 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:54.659825 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 561/1114] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:14:54 2022] GET /identity/v3/users/89964c517aa44ec1a4f2f9b0694cdc82 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:54.665556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed32f51-6a21-47ff-b2e0-7a30ba83a7e0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:54.710996 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:54.766164 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed32f51-6a21-47ff-b2e0-7a30ba83a7e0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:54.798720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1f0f6-be5a-476f-a673-433efa3264dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:54.799953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1f0f6-be5a-476f-a673-433efa3264dc None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:54.800226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1f0f6-be5a-476f-a673-433efa3264dc None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:54.800491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1f0f6-be5a-476f-a673-433efa3264dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:54.866950 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c1f0f6-be5a-476f-a673-433efa3264dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:54.868449 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c1f0f6-be5a-476f-a673-433efa3264dc None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:54.909337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:54.969386 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 554/1115] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:14:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 16:14:54.975954 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd5c88-d039-4b90-adf6-f5bf1bcbef06 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:55.075628 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cd5c88-d039-4b90-adf6-f5bf1bcbef06 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:55.109220 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f44147-f7b2-4063-a28c-3377af607f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:55.110076 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f44147-f7b2-4063-a28c-3377af607f39 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:55.110359 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f44147-f7b2-4063-a28c-3377af607f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:55.110638 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f44147-f7b2-4063-a28c-3377af607f39 None admin] PATH_INFO: `/v3/projects/26f5fb597b6943c7a1139be24fa10801` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:55.111962 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f44147-f7b2-4063-a28c-3377af607f39 None admin] RBAC: Authorizing `identity:get_project(project_id=26f5fb597b6943c7a1139be24fa10801)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:55.113408 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f44147-f7b2-4063-a28c-3377af607f39 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:55.115105 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 562/1116] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:14:54 2022] GET /identity/v3/projects/26f5fb597b6943c7a1139be24fa10801 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:55.121243 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1f0f6-be5a-476f-a673-433efa3264dc None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:55.222415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1f0f6-be5a-476f-a673-433efa3264dc None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:55.257209 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f3fa9-ad5d-4912-a35a-f9b7dfe2d7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:55.257890 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f3fa9-ad5d-4912-a35a-f9b7dfe2d7e7 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:55.258118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f3fa9-ad5d-4912-a35a-f9b7dfe2d7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:55.258442 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f3fa9-ad5d-4912-a35a-f9b7dfe2d7e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:55.259304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f3fa9-ad5d-4912-a35a-f9b7dfe2d7e7 None admin] RBAC: Adding query filter params (role.id=cfeec012f4ec48138c0e637e9a7fde61, scope.project.id=26f5fb597b6943c7a1139be24fa10801, user.id=89964c517aa44ec1a4f2f9b0694cdc82) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:14:55.259648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f3fa9-ad5d-4912-a35a-f9b7dfe2d7e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:55.261074 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f3fa9-ad5d-4912-a35a-f9b7dfe2d7e7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:55.266384 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 555/1117] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:14:55 2022] GET /identity/v3/role_assignments?user.id=89964c517aa44ec1a4f2f9b0694cdc82&scope.project.id=26f5fb597b6943c7a1139be24fa10801&role.id=cfeec012f4ec48138c0e637e9a7fde61 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:57.308894 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941e428-9b40-4a30-ac76-d0650199ffb8 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:57.309431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941e428-9b40-4a30-ac76-d0650199ffb8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:57.309431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941e428-9b40-4a30-ac76-d0650199ffb8 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:57.310747 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 563/1118] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:14:57.316670 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644493f9-d519-406d-a9bd-3ddf6bcd4761 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:57.317111 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644493f9-d519-406d-a9bd-3ddf6bcd4761 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:57.317247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644493f9-d519-406d-a9bd-3ddf6bcd4761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:57.328205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:57.334596 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-644493f9-d519-406d-a9bd-3ddf6bcd4761 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:14:57.335825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-644493f9-d519-406d-a9bd-3ddf6bcd4761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:14:57.400764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 556/1119] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:14:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:14:57.423357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941e428-9b40-4a30-ac76-d0650199ffb8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:57.475159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:57.525056 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941e428-9b40-4a30-ac76-d0650199ffb8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:57.557864 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36664a-25b9-4100-956b-320a52132757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:57.558711 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36664a-25b9-4100-956b-320a52132757 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:57.559042 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36664a-25b9-4100-956b-320a52132757 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:57.559270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36664a-25b9-4100-956b-320a52132757 None admin] PATH_INFO: `/v3/roles/cfeec012f4ec48138c0e637e9a7fde61` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:57.560753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36664a-25b9-4100-956b-320a52132757 None admin] RBAC: Authorizing `identity:get_role(role_id=cfeec012f4ec48138c0e637e9a7fde61)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:57.562124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36664a-25b9-4100-956b-320a52132757 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:57.563552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 564/1120] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:14:57 2022] GET /identity/v3/roles/cfeec012f4ec48138c0e637e9a7fde61 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:57.570139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644493f9-d519-406d-a9bd-3ddf6bcd4761 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:57.666215 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644493f9-d519-406d-a9bd-3ddf6bcd4761 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:57.699765 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27d198-867a-4bd1-b8d2-5902d4d20f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:57.700628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27d198-867a-4bd1-b8d2-5902d4d20f56 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:57.700908 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27d198-867a-4bd1-b8d2-5902d4d20f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:57.701183 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e27d198-867a-4bd1-b8d2-5902d4d20f56 None admin] PATH_INFO: `/v3/users/89964c517aa44ec1a4f2f9b0694cdc82` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:57.702362 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27d198-867a-4bd1-b8d2-5902d4d20f56 None admin] RBAC: Authorizing `identity:get_user(user_id=89964c517aa44ec1a4f2f9b0694cdc82)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:57.703810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e27d198-867a-4bd1-b8d2-5902d4d20f56 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:57.705564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 557/1121] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:14:57 2022] GET /identity/v3/users/89964c517aa44ec1a4f2f9b0694cdc82 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:57.711370 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36664a-25b9-4100-956b-320a52132757 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:57.810680 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36664a-25b9-4100-956b-320a52132757 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:57.827733 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:57.844241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d5bf3-f23f-4b0c-8dc0-df4ddefc031f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:57.845089 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d5bf3-f23f-4b0c-8dc0-df4ddefc031f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:57.845400 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d5bf3-f23f-4b0c-8dc0-df4ddefc031f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:57.846671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d5bf3-f23f-4b0c-8dc0-df4ddefc031f None admin] PATH_INFO: `/v3/projects/26f5fb597b6943c7a1139be24fa10801` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:57.847900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d5bf3-f23f-4b0c-8dc0-df4ddefc031f None admin] RBAC: Authorizing `identity:get_project(project_id=26f5fb597b6943c7a1139be24fa10801)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:57.849336 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d5bf3-f23f-4b0c-8dc0-df4ddefc031f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:57.851303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 565/1122] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:14:57 2022] GET /identity/v3/projects/26f5fb597b6943c7a1139be24fa10801 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:14:57.858019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27d198-867a-4bd1-b8d2-5902d4d20f56 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:14:57.862474 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:14:57.956778 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e27d198-867a-4bd1-b8d2-5902d4d20f56 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:14:57.990236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401a77d-0456-4d9a-a496-4586a86ed2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:14:57.991121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401a77d-0456-4d9a-a496-4586a86ed2d0 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:14:57.991400 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401a77d-0456-4d9a-a496-4586a86ed2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:14:57.991773 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401a77d-0456-4d9a-a496-4586a86ed2d0 None admin] PATH_INFO: `/v3/projects/26f5fb597b6943c7a1139be24fa10801/users/89964c517aa44ec1a4f2f9b0694cdc82/roles/cfeec012f4ec48138c0e637e9a7fde61` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:14:57.994012 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401a77d-0456-4d9a-a496-4586a86ed2d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=26f5fb597b6943c7a1139be24fa10801, user_id=89964c517aa44ec1a4f2f9b0694cdc82, role_id=cfeec012f4ec48138c0e637e9a7fde61)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:14:57.995482 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401a77d-0456-4d9a-a496-4586a86ed2d0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:14:58.093517 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 558/1123] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 30 16:14:57 2022] PUT /identity/v3/projects/26f5fb597b6943c7a1139be24fa10801/users/89964c517aa44ec1a4f2f9b0694cdc82/roles/cfeec012f4ec48138c0e637e9a7fde61 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:15:00.090742 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9beb1e-5cd2-4cce-b7a6-3c4a3ee03e4b None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:00.091295 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9beb1e-5cd2-4cce-b7a6-3c4a3ee03e4b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:00.091295 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9beb1e-5cd2-4cce-b7a6-3c4a3ee03e4b None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:00.092689 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 566/1124] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:15:00.098396 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32822f63-213c-424e-9c41-c244e06d54c1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:00.098795 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32822f63-213c-424e-9c41-c244e06d54c1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:00.098951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32822f63-213c-424e-9c41-c244e06d54c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:00.116017 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-32822f63-213c-424e-9c41-c244e06d54c1 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:15:00.117184 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-32822f63-213c-424e-9c41-c244e06d54c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:15:00.175730 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:00.183773 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 559/1125] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:15:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:15:00.207332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9beb1e-5cd2-4cce-b7a6-3c4a3ee03e4b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:00.309586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9beb1e-5cd2-4cce-b7a6-3c4a3ee03e4b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:00.342093 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d305c6-8aa3-4637-8e4e-03b8c10ee423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:00.342943 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d305c6-8aa3-4637-8e4e-03b8c10ee423 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:00.343230 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d305c6-8aa3-4637-8e4e-03b8c10ee423 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:00.343534 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d305c6-8aa3-4637-8e4e-03b8c10ee423 None admin] PATH_INFO: `/v3/roles/cfeec012f4ec48138c0e637e9a7fde61` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:00.345004 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d305c6-8aa3-4637-8e4e-03b8c10ee423 None admin] RBAC: Authorizing `identity:get_role(role_id=cfeec012f4ec48138c0e637e9a7fde61)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:00.346984 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d305c6-8aa3-4637-8e4e-03b8c10ee423 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:00.348384 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 567/1126] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:15:00 2022] GET /identity/v3/roles/cfeec012f4ec48138c0e637e9a7fde61 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:00.354321 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32822f63-213c-424e-9c41-c244e06d54c1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:00.454239 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32822f63-213c-424e-9c41-c244e06d54c1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:00.490246 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd7878-f306-4bf6-94e3-258386e88cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:00.491138 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd7878-f306-4bf6-94e3-258386e88cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:00.491418 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd7878-f306-4bf6-94e3-258386e88cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:00.491738 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd7878-f306-4bf6-94e3-258386e88cd5 None admin] PATH_INFO: `/v3/users/89964c517aa44ec1a4f2f9b0694cdc82` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:00.492911 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd7878-f306-4bf6-94e3-258386e88cd5 None admin] RBAC: Authorizing `identity:get_user(user_id=89964c517aa44ec1a4f2f9b0694cdc82)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:00.494341 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd7878-f306-4bf6-94e3-258386e88cd5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:00.496069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 560/1127] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:15:00 2022] GET /identity/v3/users/89964c517aa44ec1a4f2f9b0694cdc82 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:00.502229 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d305c6-8aa3-4637-8e4e-03b8c10ee423 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:00.558415 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:00.603088 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d305c6-8aa3-4637-8e4e-03b8c10ee423 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:00.636519 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a053965f-85b7-4887-97bc-5fb8e92cddec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:00.637159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053965f-85b7-4887-97bc-5fb8e92cddec None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:00.637447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053965f-85b7-4887-97bc-5fb8e92cddec None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:00.637729 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053965f-85b7-4887-97bc-5fb8e92cddec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:00.705399 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a053965f-85b7-4887-97bc-5fb8e92cddec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:00.706853 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a053965f-85b7-4887-97bc-5fb8e92cddec None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:00.759612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:00.804598 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 568/1128] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:15:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 16:15:00.810516 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd7878-f306-4bf6-94e3-258386e88cd5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:00.872259 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:00.913529 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd7878-f306-4bf6-94e3-258386e88cd5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:00.949476 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91571cc-83a2-4c31-8e09-1bec5cb823c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:00.950326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91571cc-83a2-4c31-8e09-1bec5cb823c2 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:00.950611 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91571cc-83a2-4c31-8e09-1bec5cb823c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:00.950889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91571cc-83a2-4c31-8e09-1bec5cb823c2 None admin] PATH_INFO: `/v3/projects/26f5fb597b6943c7a1139be24fa10801` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:00.952087 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91571cc-83a2-4c31-8e09-1bec5cb823c2 None admin] RBAC: Authorizing `identity:get_project(project_id=26f5fb597b6943c7a1139be24fa10801)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:00.953516 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91571cc-83a2-4c31-8e09-1bec5cb823c2 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:00.955932 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 561/1129] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:15:00 2022] GET /identity/v3/projects/26f5fb597b6943c7a1139be24fa10801 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:00.962335 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a053965f-85b7-4887-97bc-5fb8e92cddec None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:01.068480 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a053965f-85b7-4887-97bc-5fb8e92cddec None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:01.102118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b0e161-b41f-4832-a0ec-886ff52698f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:01.102758 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0e161-b41f-4832-a0ec-886ff52698f7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:01.103038 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0e161-b41f-4832-a0ec-886ff52698f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:01.103313 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b0e161-b41f-4832-a0ec-886ff52698f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:01.104158 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b0e161-b41f-4832-a0ec-886ff52698f7 None admin] RBAC: Adding query filter params (role.id=cfeec012f4ec48138c0e637e9a7fde61, scope.project.id=26f5fb597b6943c7a1139be24fa10801, user.id=89964c517aa44ec1a4f2f9b0694cdc82) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:15:01.104466 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b0e161-b41f-4832-a0ec-886ff52698f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:01.105889 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b0e161-b41f-4832-a0ec-886ff52698f7 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:01.112108 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 569/1130] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:15:00 2022] GET /identity/v3/role_assignments?user.id=89964c517aa44ec1a4f2f9b0694cdc82&scope.project.id=26f5fb597b6943c7a1139be24fa10801&role.id=cfeec012f4ec48138c0e637e9a7fde61 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:03.198676 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48299d-71ec-4cd5-a104-68f90682fa4a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:03.199233 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48299d-71ec-4cd5-a104-68f90682fa4a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:03.199233 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48299d-71ec-4cd5-a104-68f90682fa4a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:03.200622 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 562/1131] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:15:03.206506 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f3e6f-6e56-4912-bb1e-9d67865681a5 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:03.206900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f3e6f-6e56-4912-bb1e-9d67865681a5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:03.207072 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f3e6f-6e56-4912-bb1e-9d67865681a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:03.223740 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7c4f3e6f-6e56-4912-bb1e-9d67865681a5 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:15:03.224934 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7c4f3e6f-6e56-4912-bb1e-9d67865681a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:15:03.238518 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:03.292816 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 570/1132] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:15:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:15:03.317566 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48299d-71ec-4cd5-a104-68f90682fa4a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:03.418165 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48299d-71ec-4cd5-a104-68f90682fa4a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:03.446638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:03.452073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a20012-496c-47f6-b3e8-419ee1335e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:03.453212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a20012-496c-47f6-b3e8-419ee1335e19 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:03.453212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a20012-496c-47f6-b3e8-419ee1335e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:03.453548 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a20012-496c-47f6-b3e8-419ee1335e19 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:03.455328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a20012-496c-47f6-b3e8-419ee1335e19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:03.456801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a20012-496c-47f6-b3e8-419ee1335e19 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:03.458406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 563/1133] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:15:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:03.464532 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f3e6f-6e56-4912-bb1e-9d67865681a5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:03.562053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f3e6f-6e56-4912-bb1e-9d67865681a5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:03.596097 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1fd6e2-91c9-4219-95cf-230bd68174d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:03.596755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fd6e2-91c9-4219-95cf-230bd68174d2 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:03.597037 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fd6e2-91c9-4219-95cf-230bd68174d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:03.597310 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fd6e2-91c9-4219-95cf-230bd68174d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:03.598022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1fd6e2-91c9-4219-95cf-230bd68174d2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:03.599448 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1fd6e2-91c9-4219-95cf-230bd68174d2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:03.706281 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:03.712419 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 571/1134] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 30 16:15:03 2022] POST /identity/v3/projects => generated 311 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:15:05.809129 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5e76a-1674-44e3-87ea-4bac218eca67 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:05.809669 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5e76a-1674-44e3-87ea-4bac218eca67 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:05.809669 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5e76a-1674-44e3-87ea-4bac218eca67 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:05.811048 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 564/1135] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:15:05.817341 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f92d8-564e-418f-94df-646de4beb476 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:05.817766 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f92d8-564e-418f-94df-646de4beb476 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:05.817890 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384f92d8-564e-418f-94df-646de4beb476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:05.837619 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-384f92d8-564e-418f-94df-646de4beb476 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:15:05.838811 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-384f92d8-564e-418f-94df-646de4beb476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:15:05.911089 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 572/1136] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:15:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:15:05.939848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d5e76a-1674-44e3-87ea-4bac218eca67 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:06.048922 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d5e76a-1674-44e3-87ea-4bac218eca67 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:06.083013 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbaf517-836c-4180-bc00-8ed5530dd21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:06.083759 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbaf517-836c-4180-bc00-8ed5530dd21f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:06.084040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbaf517-836c-4180-bc00-8ed5530dd21f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:06.084312 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbaf517-836c-4180-bc00-8ed5530dd21f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:06.085484 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbaf517-836c-4180-bc00-8ed5530dd21f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:06.086904 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbaf517-836c-4180-bc00-8ed5530dd21f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:06.088546 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 565/1137] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:15:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:06.094281 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f92d8-564e-418f-94df-646de4beb476 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:06.192610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384f92d8-564e-418f-94df-646de4beb476 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:06.220905 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:06.226430 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44983c-c2c6-40b0-a4ec-756ec4ab662c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:06.227097 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44983c-c2c6-40b0-a4ec-756ec4ab662c None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:06.227383 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44983c-c2c6-40b0-a4ec-756ec4ab662c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:06.227686 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44983c-c2c6-40b0-a4ec-756ec4ab662c None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:06.228426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44983c-c2c6-40b0-a4ec-756ec4ab662c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:06.229866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b44983c-c2c6-40b0-a4ec-756ec4ab662c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:06.373432 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 573/1138] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 30 16:15:06 2022] POST /identity/v3/users => generated 286 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:15:08.448000 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16eddc8-cd95-4ff9-9793-abac2e82bff2 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:08.448553 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16eddc8-cd95-4ff9-9793-abac2e82bff2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:08.448553 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16eddc8-cd95-4ff9-9793-abac2e82bff2 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:08.449916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 566/1139] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:15:08.456533 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf4de9-aaa0-4343-b269-375115649258 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:08.456958 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf4de9-aaa0-4343-b269-375115649258 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:08.457093 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaf4de9-aaa0-4343-b269-375115649258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:08.474079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-afaf4de9-aaa0-4343-b269-375115649258 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:15:08.475255 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-afaf4de9-aaa0-4343-b269-375115649258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:15:08.541608 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 574/1140] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:15:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:15:08.566100 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16eddc8-cd95-4ff9-9793-abac2e82bff2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:08.632818 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:08.667480 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16eddc8-cd95-4ff9-9793-abac2e82bff2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:08.701759 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ac74c-3695-4923-b968-6dafddeeee7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:08.702468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ac74c-3695-4923-b968-6dafddeeee7f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:08.702742 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ac74c-3695-4923-b968-6dafddeeee7f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:08.703008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ac74c-3695-4923-b968-6dafddeeee7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:08.713770 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ac74c-3695-4923-b968-6dafddeeee7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:15:08.721609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-7e5ac74c-3695-4923-b968-6dafddeeee7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:15:08.721609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:15:08.721609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:15:08.721609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:15:08.721609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:15:08.721609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:15:08.721609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:15:08.721609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:15:08.721609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:15:08.721609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:15:08.721609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:15:08.721609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:15:08.721609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:15:08.721609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:15:08.721609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:15:08.721609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:15:08.722763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:15:08.722763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:15:08.722763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:15:08.722763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:15:08.722763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:15:08.722763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:15:08.722763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:15:08.722763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:15:08.722763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:15:08.722763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:15:08.722763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:15:08.722763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:15:08.722763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:15:08.722763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:15:08.722763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:15:08.722763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:15:08.723712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:15:08.723712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:15:08.723712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:15:08.723712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:15:08.723712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:15:08.723712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:15:08.723712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:15:08.723712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:15:08.723712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:15:08.723712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:15:08.723712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:15:08.723712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:15:08.723712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 567/1141] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:15:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:15:08.728912 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf4de9-aaa0-4343-b269-375115649258 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:08.826289 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaf4de9-aaa0-4343-b269-375115649258 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:08.842310 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:08.859689 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32f634-e6a7-4b56-b89d-4a4aa6c06fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:08.860874 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32f634-e6a7-4b56-b89d-4a4aa6c06fba None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:08.861420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32f634-e6a7-4b56-b89d-4a4aa6c06fba None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:08.861698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32f634-e6a7-4b56-b89d-4a4aa6c06fba None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:08.862438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32f634-e6a7-4b56-b89d-4a4aa6c06fba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:15:08.862789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32f634-e6a7-4b56-b89d-4a4aa6c06fba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:08.864186 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32f634-e6a7-4b56-b89d-4a4aa6c06fba None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:08.872160 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 575/1142] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:15:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:08.878487 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ac74c-3695-4923-b968-6dafddeeee7f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:08.978836 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ac74c-3695-4923-b968-6dafddeeee7f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:09.007585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:09.016691 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3333f-5c23-48bc-b04e-05168cd50952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:09.017901 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3333f-5c23-48bc-b04e-05168cd50952 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:09.021098 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3333f-5c23-48bc-b04e-05168cd50952 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:09.021098 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a3333f-5c23-48bc-b04e-05168cd50952 None admin] PATH_INFO: `/v3/users/e5e3616fb37547e4a4fbf4978f05e3fb` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:09.032509 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3333f-5c23-48bc-b04e-05168cd50952 None admin] RBAC: Authorizing `identity:get_user(user_id=e5e3616fb37547e4a4fbf4978f05e3fb)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:09.033968 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a3333f-5c23-48bc-b04e-05168cd50952 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:09.035779 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 568/1143] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:15:08 2022] GET /identity/v3/users/e5e3616fb37547e4a4fbf4978f05e3fb => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:09.041653 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32f634-e6a7-4b56-b89d-4a4aa6c06fba None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:09.141854 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32f634-e6a7-4b56-b89d-4a4aa6c06fba None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:09.175809 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ad183-5305-4d68-b92a-be15f322d8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:09.176450 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ad183-5305-4d68-b92a-be15f322d8ed None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:09.176740 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ad183-5305-4d68-b92a-be15f322d8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:09.177018 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ad183-5305-4d68-b92a-be15f322d8ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:09.221559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:09.243372 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ad183-5305-4d68-b92a-be15f322d8ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:09.244869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ad183-5305-4d68-b92a-be15f322d8ed None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:09.341589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 576/1144] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:15:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 16:15:09.348244 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3333f-5c23-48bc-b04e-05168cd50952 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:09.446579 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a3333f-5c23-48bc-b04e-05168cd50952 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:09.479325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b4d0b-4bef-4ecd-b4a3-66f0a81767d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:09.480229 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b4d0b-4bef-4ecd-b4a3-66f0a81767d0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:09.480510 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b4d0b-4bef-4ecd-b4a3-66f0a81767d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:09.480784 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b4d0b-4bef-4ecd-b4a3-66f0a81767d0 None admin] PATH_INFO: `/v3/projects/7027427317c941d48976428cf1a8302f` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:09.482075 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b4d0b-4bef-4ecd-b4a3-66f0a81767d0 None admin] RBAC: Authorizing `identity:get_project(project_id=7027427317c941d48976428cf1a8302f)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:09.483527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b4d0b-4bef-4ecd-b4a3-66f0a81767d0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:09.485262 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 569/1145] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:15:09 2022] GET /identity/v3/projects/7027427317c941d48976428cf1a8302f => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:09.492244 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ad183-5305-4d68-b92a-be15f322d8ed None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:09.567742 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:09.588802 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ad183-5305-4d68-b92a-be15f322d8ed None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:09.622266 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9084286-7972-4e0a-a060-8d576094d327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:09.622923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9084286-7972-4e0a-a060-8d576094d327 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:09.623204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9084286-7972-4e0a-a060-8d576094d327 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:09.623502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9084286-7972-4e0a-a060-8d576094d327 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:09.624333 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9084286-7972-4e0a-a060-8d576094d327 None admin] RBAC: Adding query filter params (role.id=8acc34a163874a60b03728b814eaf075, scope.project.id=7027427317c941d48976428cf1a8302f, user.id=e5e3616fb37547e4a4fbf4978f05e3fb) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:15:09.624648 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9084286-7972-4e0a-a060-8d576094d327 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:09.626286 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9084286-7972-4e0a-a060-8d576094d327 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:09.631656 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 577/1146] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:15:09 2022] GET /identity/v3/role_assignments?user.id=e5e3616fb37547e4a4fbf4978f05e3fb&scope.project.id=7027427317c941d48976428cf1a8302f&role.id=8acc34a163874a60b03728b814eaf075 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:11.676193 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839583b-033a-4b37-8a6f-78e5d18de128 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:11.676739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839583b-033a-4b37-8a6f-78e5d18de128 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:11.676739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839583b-033a-4b37-8a6f-78e5d18de128 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:11.678126 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 570/1147] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:15:11.683612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236206e-252e-4899-ae4a-f343d6751689 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:11.684081 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236206e-252e-4899-ae4a-f343d6751689 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:11.684170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236206e-252e-4899-ae4a-f343d6751689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:11.703718 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0236206e-252e-4899-ae4a-f343d6751689 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:15:11.705507 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0236206e-252e-4899-ae4a-f343d6751689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:15:11.772749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 578/1148] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:15:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:15:11.795836 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839583b-033a-4b37-8a6f-78e5d18de128 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:11.849811 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:11.898413 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839583b-033a-4b37-8a6f-78e5d18de128 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:11.932303 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a7aa8-8348-4f3e-a28b-8e5941f2aab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:11.933026 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a7aa8-8348-4f3e-a28b-8e5941f2aab1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:11.933308 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a7aa8-8348-4f3e-a28b-8e5941f2aab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:11.933583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a7aa8-8348-4f3e-a28b-8e5941f2aab1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:11.944839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a7aa8-8348-4f3e-a28b-8e5941f2aab1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:15:11.952547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e00a7aa8-8348-4f3e-a28b-8e5941f2aab1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:15:11.952547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:15:11.952547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:15:11.952547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:15:11.952547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:15:11.952547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:15:11.952547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:15:11.952547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:15:11.952547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:15:11.952547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:15:11.952547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:15:11.952547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:15:11.952547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:15:11.952547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:15:11.952547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:15:11.952547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:15:11.953683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:15:11.953683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:15:11.953683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:15:11.953683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:15:11.953683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:15:11.953683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:15:11.953683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:15:11.953683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:15:11.953683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:15:11.953683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:15:11.953683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:15:11.953683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:15:11.953683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:15:11.953683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:15:11.953683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:15:11.953683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:15:11.954880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:15:11.954880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:15:11.954880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:15:11.954880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:15:11.954880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:15:11.954880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:15:11.954880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:15:11.954880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:15:11.954880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:15:11.954880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:15:11.954880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:15:11.954880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:15:11.954880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 571/1149] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:15:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:15:11.960367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0236206e-252e-4899-ae4a-f343d6751689 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:12.028855 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:12.057588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0236206e-252e-4899-ae4a-f343d6751689 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:12.091973 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefbe792-5d9e-4780-8e23-cccb6e488c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:12.092627 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbe792-5d9e-4780-8e23-cccb6e488c71 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:12.092907 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbe792-5d9e-4780-8e23-cccb6e488c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:12.093182 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbe792-5d9e-4780-8e23-cccb6e488c71 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:12.093914 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefbe792-5d9e-4780-8e23-cccb6e488c71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:15:12.094219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefbe792-5d9e-4780-8e23-cccb6e488c71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:12.096548 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefbe792-5d9e-4780-8e23-cccb6e488c71 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:12.104545 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 579/1150] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:15:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:12.111610 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a7aa8-8348-4f3e-a28b-8e5941f2aab1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:12.208824 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a7aa8-8348-4f3e-a28b-8e5941f2aab1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:12.222119 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:12.242789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b9bd8-988c-4d26-b6a0-ce3ca8ee3ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:12.243684 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b9bd8-988c-4d26-b6a0-ce3ca8ee3ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:12.243964 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b9bd8-988c-4d26-b6a0-ce3ca8ee3ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:12.244231 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b9bd8-988c-4d26-b6a0-ce3ca8ee3ec4 None admin] PATH_INFO: `/v3/users/e5e3616fb37547e4a4fbf4978f05e3fb` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:12.245392 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b9bd8-988c-4d26-b6a0-ce3ca8ee3ec4 None admin] RBAC: Authorizing `identity:get_user(user_id=e5e3616fb37547e4a4fbf4978f05e3fb)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:12.246819 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b9bd8-988c-4d26-b6a0-ce3ca8ee3ec4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:12.248467 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 572/1151] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:15:12 2022] GET /identity/v3/users/e5e3616fb37547e4a4fbf4978f05e3fb => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:12.254540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefbe792-5d9e-4780-8e23-cccb6e488c71 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:12.356684 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefbe792-5d9e-4780-8e23-cccb6e488c71 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:12.389706 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f22be37-eaa7-4d36-ad37-7ef0808c593a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:12.390556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22be37-eaa7-4d36-ad37-7ef0808c593a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:12.390843 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22be37-eaa7-4d36-ad37-7ef0808c593a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:12.391123 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22be37-eaa7-4d36-ad37-7ef0808c593a None admin] PATH_INFO: `/v3/projects/7027427317c941d48976428cf1a8302f` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:12.392421 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f22be37-eaa7-4d36-ad37-7ef0808c593a None admin] RBAC: Authorizing `identity:get_project(project_id=7027427317c941d48976428cf1a8302f)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:12.393917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f22be37-eaa7-4d36-ad37-7ef0808c593a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:12.395887 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 580/1152] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:15:12 2022] GET /identity/v3/projects/7027427317c941d48976428cf1a8302f => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:12.401898 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b9bd8-988c-4d26-b6a0-ce3ca8ee3ec4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:12.502316 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b9bd8-988c-4d26-b6a0-ce3ca8ee3ec4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:12.536664 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2aafd7-8a31-4c9d-a487-4cdcafc1aa16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:12.537621 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2aafd7-8a31-4c9d-a487-4cdcafc1aa16 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:12.537904 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2aafd7-8a31-4c9d-a487-4cdcafc1aa16 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:12.538178 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2aafd7-8a31-4c9d-a487-4cdcafc1aa16 None admin] PATH_INFO: `/v3/projects/7027427317c941d48976428cf1a8302f/users/e5e3616fb37547e4a4fbf4978f05e3fb/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:12.540153 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2aafd7-8a31-4c9d-a487-4cdcafc1aa16 None admin] RBAC: Authorizing `identity:create_grant(project_id=7027427317c941d48976428cf1a8302f, user_id=e5e3616fb37547e4a4fbf4978f05e3fb, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:12.541590 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2aafd7-8a31-4c9d-a487-4cdcafc1aa16 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:12.623285 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 573/1153] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 30 16:15:12 2022] PUT /identity/v3/projects/7027427317c941d48976428cf1a8302f/users/e5e3616fb37547e4a4fbf4978f05e3fb/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:15:14.661081 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e615128-377a-4496-867b-38e4ed4f37b0 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:14.661646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e615128-377a-4496-867b-38e4ed4f37b0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:14.661646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e615128-377a-4496-867b-38e4ed4f37b0 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:14.662980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 581/1154] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:15:14.668102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6820de-f7bc-43cc-8baf-489bd7b4d57d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:14.668490 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6820de-f7bc-43cc-8baf-489bd7b4d57d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:14.668663 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6820de-f7bc-43cc-8baf-489bd7b4d57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:14.685772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6d6820de-f7bc-43cc-8baf-489bd7b4d57d None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:15:14.686988 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6d6820de-f7bc-43cc-8baf-489bd7b4d57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:15:14.721812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:14.754953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 574/1155] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:15:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:15:14.778977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e615128-377a-4496-867b-38e4ed4f37b0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:14.782038 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:14.881405 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e615128-377a-4496-867b-38e4ed4f37b0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:14.915218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd6edf-0391-424e-87e1-113bad1f9c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:14.915950 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd6edf-0391-424e-87e1-113bad1f9c05 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:14.916228 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd6edf-0391-424e-87e1-113bad1f9c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:14.916502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd6edf-0391-424e-87e1-113bad1f9c05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:14.927228 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cd6edf-0391-424e-87e1-113bad1f9c05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:15:14.934803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-96cd6edf-0391-424e-87e1-113bad1f9c05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:15:14.934803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:15:14.934803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:15:14.934803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:15:14.934803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:15:14.934803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:15:14.934803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:15:14.934803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:15:14.934803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:15:14.934803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:15:14.934803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:15:14.934803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:15:14.934803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:15:14.934803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:15:14.934803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:15:14.934803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:15:14.935784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:15:14.935784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:15:14.935784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:15:14.935784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:15:14.935784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:15:14.935784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:15:14.935784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:15:14.935784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:15:14.935784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:15:14.935784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:15:14.935784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:15:14.935784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:15:14.935784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:15:14.935784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:15:14.935784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:15:14.935784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:15:14.936631 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:15:14.936631 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:15:14.936631 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:15:14.936631 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:15:14.936631 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:15:14.936631 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:15:14.936631 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:15:14.936631 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:15:14.936631 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:15:14.936631 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:15:14.936631 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:15:14.936631 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:15:14.936631 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 582/1156] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:15:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:15:14.942861 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6820de-f7bc-43cc-8baf-489bd7b4d57d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:15.045298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6820de-f7bc-43cc-8baf-489bd7b4d57d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:15.080168 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b6d3bf-046e-4d48-8b37-ab0dad9d099e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:15.080826 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6d3bf-046e-4d48-8b37-ab0dad9d099e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:15.081110 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6d3bf-046e-4d48-8b37-ab0dad9d099e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:15.081385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6d3bf-046e-4d48-8b37-ab0dad9d099e None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:15.082122 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b6d3bf-046e-4d48-8b37-ab0dad9d099e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:15:15.082427 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b6d3bf-046e-4d48-8b37-ab0dad9d099e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:15.083938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b6d3bf-046e-4d48-8b37-ab0dad9d099e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:15.092001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 575/1157] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:15:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:15.099006 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd6edf-0391-424e-87e1-113bad1f9c05 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:15.161225 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:15.198724 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cd6edf-0391-424e-87e1-113bad1f9c05 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:15.231701 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9758e-bee5-4c62-a2cc-bcd0cb32b892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:15.232679 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9758e-bee5-4c62-a2cc-bcd0cb32b892 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:15.232964 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9758e-bee5-4c62-a2cc-bcd0cb32b892 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:15.233247 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9758e-bee5-4c62-a2cc-bcd0cb32b892 None admin] PATH_INFO: `/v3/users/e5e3616fb37547e4a4fbf4978f05e3fb` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:15.234435 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb9758e-bee5-4c62-a2cc-bcd0cb32b892 None admin] RBAC: Authorizing `identity:get_user(user_id=e5e3616fb37547e4a4fbf4978f05e3fb)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:15.236580 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb9758e-bee5-4c62-a2cc-bcd0cb32b892 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:15.238257 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 583/1158] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:15:15 2022] GET /identity/v3/users/e5e3616fb37547e4a4fbf4978f05e3fb => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:15.244144 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b6d3bf-046e-4d48-8b37-ab0dad9d099e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:15.275159 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:15.342484 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b6d3bf-046e-4d48-8b37-ab0dad9d099e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:15.377105 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9421b976-79e6-4b6c-b7cd-2a173d358fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:15.377748 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421b976-79e6-4b6c-b7cd-2a173d358fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:15.378033 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421b976-79e6-4b6c-b7cd-2a173d358fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:15.378313 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9421b976-79e6-4b6c-b7cd-2a173d358fd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:15.444803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9421b976-79e6-4b6c-b7cd-2a173d358fd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:15.446303 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9421b976-79e6-4b6c-b7cd-2a173d358fd5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:15.475426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:15.544802 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 576/1159] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:15:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 16:15:15.550980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9758e-bee5-4c62-a2cc-bcd0cb32b892 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:15.650125 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb9758e-bee5-4c62-a2cc-bcd0cb32b892 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:15.677952 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:15.683576 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd7420-7b72-4e23-9bc2-5f1d153e4e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:15.684428 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd7420-7b72-4e23-9bc2-5f1d153e4e80 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:15.684712 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd7420-7b72-4e23-9bc2-5f1d153e4e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:15.684993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd7420-7b72-4e23-9bc2-5f1d153e4e80 None admin] PATH_INFO: `/v3/projects/7027427317c941d48976428cf1a8302f` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:15.686186 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd7420-7b72-4e23-9bc2-5f1d153e4e80 None admin] RBAC: Authorizing `identity:get_project(project_id=7027427317c941d48976428cf1a8302f)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:15.687685 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd7420-7b72-4e23-9bc2-5f1d153e4e80 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:15.689412 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 584/1160] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:15:15 2022] GET /identity/v3/projects/7027427317c941d48976428cf1a8302f => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:15.695433 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9421b976-79e6-4b6c-b7cd-2a173d358fd5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:15.796016 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9421b976-79e6-4b6c-b7cd-2a173d358fd5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:15.823885 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:15.829469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7cdeca-959b-444e-88fd-6bdd48564d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:15.830114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cdeca-959b-444e-88fd-6bdd48564d99 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:15.830399 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cdeca-959b-444e-88fd-6bdd48564d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:15.830679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7cdeca-959b-444e-88fd-6bdd48564d99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:15.831581 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7cdeca-959b-444e-88fd-6bdd48564d99 None admin] RBAC: Adding query filter params (role.id=8acc34a163874a60b03728b814eaf075, scope.project.id=7027427317c941d48976428cf1a8302f, user.id=e5e3616fb37547e4a4fbf4978f05e3fb) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:15:15.843231 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7cdeca-959b-444e-88fd-6bdd48564d99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:15.843231 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7cdeca-959b-444e-88fd-6bdd48564d99 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:15.843231 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 577/1161] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:15:15 2022] GET /identity/v3/role_assignments?user.id=e5e3616fb37547e4a4fbf4978f05e3fb&scope.project.id=7027427317c941d48976428cf1a8302f&role.id=8acc34a163874a60b03728b814eaf075 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:15:17.920674 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50e6d8-a73d-4022-b8b1-c0a380e62bf6 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:17.921231 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50e6d8-a73d-4022-b8b1-c0a380e62bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:17.921231 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50e6d8-a73d-4022-b8b1-c0a380e62bf6 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:17.923106 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 585/1162] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:15:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:15:17.928393 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd3339-f19a-4496-9d73-a4b64e1d16b4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:17.928830 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd3339-f19a-4496-9d73-a4b64e1d16b4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:17.928930 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd3339-f19a-4496-9d73-a4b64e1d16b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:17.947125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f3bd3339-f19a-4496-9d73-a4b64e1d16b4 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:15:17.948420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f3bd3339-f19a-4496-9d73-a4b64e1d16b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:15:18.014556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 578/1163] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:15:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:15:18.037923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50e6d8-a73d-4022-b8b1-c0a380e62bf6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:18.146577 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50e6d8-a73d-4022-b8b1-c0a380e62bf6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:18.179754 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6a47e-6484-41e9-b1ba-59e7cc719156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:18.180318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6a47e-6484-41e9-b1ba-59e7cc719156 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:18.180598 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6a47e-6484-41e9-b1ba-59e7cc719156 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:18.180875 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6a47e-6484-41e9-b1ba-59e7cc719156 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:18.244708 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:18.248022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb6a47e-6484-41e9-b1ba-59e7cc719156 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:18.249480 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb6a47e-6484-41e9-b1ba-59e7cc719156 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:18.348816 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 586/1164] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:15:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 16:15:18.354849 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd3339-f19a-4496-9d73-a4b64e1d16b4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:18.458057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd3339-f19a-4496-9d73-a4b64e1d16b4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:18.482939 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:18.492034 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6471e-8f8d-47ed-b4e3-01e1012b1b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:18.495323 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6471e-8f8d-47ed-b4e3-01e1012b1b00 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:18.495323 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6471e-8f8d-47ed-b4e3-01e1012b1b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:18.495323 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6471e-8f8d-47ed-b4e3-01e1012b1b00 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:18.499875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca6471e-8f8d-47ed-b4e3-01e1012b1b00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:18.501896 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca6471e-8f8d-47ed-b4e3-01e1012b1b00 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:18.516678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-8ca6471e-8f8d-47ed-b4e3-01e1012b1b00 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 30 16:15:18.516678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:15:18.516678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 30 16:15:18.516678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 30 16:15:18.516678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:15:18.516678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:15:18.516678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:15:18.516678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:15:18.516678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 30 16:15:18.516678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:15:18.516678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:15:18.516678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:15:18.516678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:15:18.516678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:15:18.516678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:15:18.516678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:15:18.516678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:15:18.517957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:15:18.517957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:15:18.517957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:15:18.517957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:15:18.517957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:15:18.517957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:15:18.517957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:15:18.517957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:15:18.517957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:15:18.517957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:15:18.517957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:15:18.517957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:15:18.517957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:15:18.517957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:15:18.517957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:15:18.517957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:15:18.519208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:15:18.519208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:15:18.519208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:15:18.519208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:15:18.519208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:15:18.519208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:15:18.519208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:15:18.519208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:15:18.519208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:15:18.519208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:15:18.519208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 30 16:15:18.519208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:15:18.519208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 30 16:15:18.519208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:15:18.519208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 579/1165] 10.222.0.67 () {56 vars in 1117 bytes} [Thu Jun 30 16:15:18 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:15:18.524347 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6a47e-6484-41e9-b1ba-59e7cc719156 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:18.617364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:18.621063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6a47e-6484-41e9-b1ba-59e7cc719156 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:18.654534 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977dc28-8eef-470c-b931-f246e8c3c14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:18.655187 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977dc28-8eef-470c-b931-f246e8c3c14e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:18.655495 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977dc28-8eef-470c-b931-f246e8c3c14e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:18.655771 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977dc28-8eef-470c-b931-f246e8c3c14e None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:18.656543 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b977dc28-8eef-470c-b931-f246e8c3c14e None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:15:18.656847 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b977dc28-8eef-470c-b931-f246e8c3c14e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:18.658234 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b977dc28-8eef-470c-b931-f246e8c3c14e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:18.665239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 587/1166] 10.222.0.67 () {56 vars in 1115 bytes} [Thu Jun 30 16:15:18 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:20.718726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afeaf3-6fbf-4069-bb45-1368b0625d75 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:20.719436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afeaf3-6fbf-4069-bb45-1368b0625d75 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:20.719436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25afeaf3-6fbf-4069-bb45-1368b0625d75 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:20.720774 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 580/1167] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:15:20.725922 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338fe4a-d727-4278-9114-3fb719492891 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:20.726340 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338fe4a-d727-4278-9114-3fb719492891 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:20.726624 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338fe4a-d727-4278-9114-3fb719492891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:20.743570 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0338fe4a-d727-4278-9114-3fb719492891 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:15:20.744764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0338fe4a-d727-4278-9114-3fb719492891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:15:20.810729 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 588/1168] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:15:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:15:20.835205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afeaf3-6fbf-4069-bb45-1368b0625d75 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:20.936006 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25afeaf3-6fbf-4069-bb45-1368b0625d75 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:20.970579 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baee404-d8ee-40fe-8f85-b650a3586162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:20.971163 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baee404-d8ee-40fe-8f85-b650a3586162 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:20.971444 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baee404-d8ee-40fe-8f85-b650a3586162 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:20.971754 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baee404-d8ee-40fe-8f85-b650a3586162 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:20.972525 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baee404-d8ee-40fe-8f85-b650a3586162 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:20.973910 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baee404-d8ee-40fe-8f85-b650a3586162 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:21.059085 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 581/1169] 10.222.0.67 () {60 vars in 1138 bytes} [Thu Jun 30 16:15:20 2022] POST /identity/v3/domains => generated 267 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:15:23.285272 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2995a7-0fdf-4315-8af3-2806c94002cc None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:23.285782 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2995a7-0fdf-4315-8af3-2806c94002cc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:23.285782 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2995a7-0fdf-4315-8af3-2806c94002cc None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:23.287183 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 589/1170] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:15:23.292694 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615771f-e7b5-43de-9ba3-9f99dbf8ab41 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:23.293131 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615771f-e7b5-43de-9ba3-9f99dbf8ab41 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:23.293240 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615771f-e7b5-43de-9ba3-9f99dbf8ab41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:23.309742 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1615771f-e7b5-43de-9ba3-9f99dbf8ab41 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:15:23.310938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1615771f-e7b5-43de-9ba3-9f99dbf8ab41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:15:23.353526 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:23.377596 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 582/1171] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:15:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:15:23.403278 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2995a7-0fdf-4315-8af3-2806c94002cc None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:23.498722 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:23.511458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2995a7-0fdf-4315-8af3-2806c94002cc None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:23.545702 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fadb30-b304-49a3-aa2f-8007ce894531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:23.546487 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fadb30-b304-49a3-aa2f-8007ce894531 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:23.546796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fadb30-b304-49a3-aa2f-8007ce894531 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:23.547076 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fadb30-b304-49a3-aa2f-8007ce894531 None admin] PATH_INFO: `/v3/domains/91405799dd284b80be794cd4bab6dbb1` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:23.557367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fadb30-b304-49a3-aa2f-8007ce894531 None admin] RBAC: Authorizing `identity:get_domain(domain_id=91405799dd284b80be794cd4bab6dbb1)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:23.558811 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fadb30-b304-49a3-aa2f-8007ce894531 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:23.560470 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 590/1172] 10.222.0.67 () {56 vars in 1183 bytes} [Thu Jun 30 16:15:23 2022] GET /identity/v3/domains/91405799dd284b80be794cd4bab6dbb1 => generated 267 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:23.567282 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1615771f-e7b5-43de-9ba3-9f99dbf8ab41 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:23.665820 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1615771f-e7b5-43de-9ba3-9f99dbf8ab41 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:23.699780 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a9506-89cd-43a3-a3b1-aa879ae1b5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:23.700476 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a9506-89cd-43a3-a3b1-aa879ae1b5f9 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:23.700764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a9506-89cd-43a3-a3b1-aa879ae1b5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:23.701040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a9506-89cd-43a3-a3b1-aa879ae1b5f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:23.701757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a9506-89cd-43a3-a3b1-aa879ae1b5f9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:23.704029 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a9506-89cd-43a3-a3b1-aa879ae1b5f9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:23.819451 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 583/1173] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 30 16:15:23 2022] POST /identity/v3/projects => generated 361 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:15:25.904330 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1672ec5-e021-4dd3-9ea3-a7f7614c0d88 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:25.904922 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1672ec5-e021-4dd3-9ea3-a7f7614c0d88 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:25.904922 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1672ec5-e021-4dd3-9ea3-a7f7614c0d88 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:25.907006 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 591/1174] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:15:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:15:25.912284 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca087b-3d5c-4d4d-ade1-ce4e420145d1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:25.912707 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca087b-3d5c-4d4d-ade1-ce4e420145d1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:25.912846 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca087b-3d5c-4d4d-ade1-ce4e420145d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:25.930236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-11ca087b-3d5c-4d4d-ade1-ce4e420145d1 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:15:25.931446 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-11ca087b-3d5c-4d4d-ade1-ce4e420145d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:15:25.953997 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:25.998838 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 584/1175] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:15:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:15:26.024741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1672ec5-e021-4dd3-9ea3-a7f7614c0d88 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:26.127190 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1672ec5-e021-4dd3-9ea3-a7f7614c0d88 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:26.164050 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9323b-cf7b-4127-ac5f-a66a7d9613fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:26.164865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9323b-cf7b-4127-ac5f-a66a7d9613fa None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:26.165146 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9323b-cf7b-4127-ac5f-a66a7d9613fa None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:26.165454 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9323b-cf7b-4127-ac5f-a66a7d9613fa None admin] PATH_INFO: `/v3/domains/91405799dd284b80be794cd4bab6dbb1` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:26.166632 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa9323b-cf7b-4127-ac5f-a66a7d9613fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=91405799dd284b80be794cd4bab6dbb1)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:26.168119 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa9323b-cf7b-4127-ac5f-a66a7d9613fa None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:26.169774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 592/1176] 10.222.0.67 () {56 vars in 1183 bytes} [Thu Jun 30 16:15:26 2022] GET /identity/v3/domains/91405799dd284b80be794cd4bab6dbb1 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:26.175808 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca087b-3d5c-4d4d-ade1-ce4e420145d1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:26.273458 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca087b-3d5c-4d4d-ade1-ce4e420145d1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:26.307059 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e549716-5abe-475e-819d-90b42ac84295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:26.307787 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e549716-5abe-475e-819d-90b42ac84295 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:26.308072 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e549716-5abe-475e-819d-90b42ac84295 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:26.308345 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e549716-5abe-475e-819d-90b42ac84295 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:26.309092 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e549716-5abe-475e-819d-90b42ac84295 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:26.310600 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e549716-5abe-475e-819d-90b42ac84295 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:26.429145 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 585/1177] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 30 16:15:26 2022] POST /identity/v3/users => generated 311 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:15:28.542739 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bb8f4-56fe-4a11-af8c-4b319adaaa46 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:28.543304 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bb8f4-56fe-4a11-af8c-4b319adaaa46 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:28.543304 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bb8f4-56fe-4a11-af8c-4b319adaaa46 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:28.544682 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 593/1178] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:15:28.550555 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673032bf-caeb-4be0-bf72-64460d1f22e2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:28.550999 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673032bf-caeb-4be0-bf72-64460d1f22e2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:28.551106 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673032bf-caeb-4be0-bf72-64460d1f22e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:28.568245 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-673032bf-caeb-4be0-bf72-64460d1f22e2 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:15:28.569435 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-673032bf-caeb-4be0-bf72-64460d1f22e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:15:28.578840 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:28.636835 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 586/1179] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:15:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:15:28.661369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bb8f4-56fe-4a11-af8c-4b319adaaa46 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:28.683601 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:28.762396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bb8f4-56fe-4a11-af8c-4b319adaaa46 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:28.797594 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6e0a0-7feb-4ec2-bfd2-2bc0003f292c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:28.798270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6e0a0-7feb-4ec2-bfd2-2bc0003f292c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:28.798587 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6e0a0-7feb-4ec2-bfd2-2bc0003f292c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:28.798941 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6e0a0-7feb-4ec2-bfd2-2bc0003f292c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:28.808759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c6e0a0-7feb-4ec2-bfd2-2bc0003f292c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:15:28.816723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-85c6e0a0-7feb-4ec2-bfd2-2bc0003f292c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:15:28.816723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:15:28.816723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:15:28.816723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:15:28.816723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:15:28.816723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:15:28.816723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:15:28.816723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:15:28.816723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:15:28.816723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:15:28.816723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:15:28.816723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:15:28.816723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:15:28.816723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:15:28.816723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:15:28.816723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:15:28.817844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:15:28.817844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:15:28.817844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:15:28.817844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:15:28.817844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:15:28.817844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:15:28.817844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:15:28.817844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:15:28.817844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:15:28.817844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:15:28.817844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:15:28.817844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:15:28.817844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:15:28.817844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:15:28.817844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:15:28.817844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:15:28.819109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:15:28.819109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:15:28.819109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:15:28.819109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:15:28.819109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:15:28.819109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:15:28.819109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:15:28.819109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:15:28.819109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:15:28.819109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:15:28.819109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:15:28.819109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:15:28.819109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 594/1180] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:15:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:15:28.824022 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673032bf-caeb-4be0-bf72-64460d1f22e2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:28.921987 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673032bf-caeb-4be0-bf72-64460d1f22e2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:28.954919 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e129ca1-8e01-44f7-994e-24eee00b50db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:28.955609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e129ca1-8e01-44f7-994e-24eee00b50db None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:28.955894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e129ca1-8e01-44f7-994e-24eee00b50db None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:28.956170 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e129ca1-8e01-44f7-994e-24eee00b50db None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:28.956905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e129ca1-8e01-44f7-994e-24eee00b50db None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:15:28.957209 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e129ca1-8e01-44f7-994e-24eee00b50db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:28.958568 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e129ca1-8e01-44f7-994e-24eee00b50db None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:28.966523 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 587/1181] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:15:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:28.972953 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6e0a0-7feb-4ec2-bfd2-2bc0003f292c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:29.054824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:29.073648 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6e0a0-7feb-4ec2-bfd2-2bc0003f292c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:29.108234 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b82683-5606-4013-ad1a-d894271f667d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:29.109126 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b82683-5606-4013-ad1a-d894271f667d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:29.109403 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b82683-5606-4013-ad1a-d894271f667d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:29.109677 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b82683-5606-4013-ad1a-d894271f667d None admin] PATH_INFO: `/v3/users/2fbd9323d27a473ca52ba437aec2aefa` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:29.122589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b82683-5606-4013-ad1a-d894271f667d None admin] RBAC: Authorizing `identity:get_user(user_id=2fbd9323d27a473ca52ba437aec2aefa)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:29.124074 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b82683-5606-4013-ad1a-d894271f667d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:29.125991 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 595/1182] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:15:28 2022] GET /identity/v3/users/2fbd9323d27a473ca52ba437aec2aefa => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:29.131798 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e129ca1-8e01-44f7-994e-24eee00b50db None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:29.141743 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:29.232600 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e129ca1-8e01-44f7-994e-24eee00b50db None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:29.265704 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b4e8e-84b5-4440-8b31-6f0585c7d0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:29.266433 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b4e8e-84b5-4440-8b31-6f0585c7d0ba None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:29.266714 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b4e8e-84b5-4440-8b31-6f0585c7d0ba None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:29.266985 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b4e8e-84b5-4440-8b31-6f0585c7d0ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:29.332595 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b4e8e-84b5-4440-8b31-6f0585c7d0ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:29.334041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b4e8e-84b5-4440-8b31-6f0585c7d0ba None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:29.341364 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:29.431433 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 588/1183] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:15:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 16:15:29.437966 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b82683-5606-4013-ad1a-d894271f667d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:29.533411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b82683-5606-4013-ad1a-d894271f667d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:29.565995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ae715-93bc-491e-814e-da1e43d735c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:29.566844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ae715-93bc-491e-814e-da1e43d735c3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:29.567129 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ae715-93bc-491e-814e-da1e43d735c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:29.567403 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ae715-93bc-491e-814e-da1e43d735c3 None admin] PATH_INFO: `/v3/projects/4a2443ef903c492b90316bf4786d5f5d` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:29.568596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ae715-93bc-491e-814e-da1e43d735c3 None admin] RBAC: Authorizing `identity:get_project(project_id=4a2443ef903c492b90316bf4786d5f5d)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:29.570031 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ae715-93bc-491e-814e-da1e43d735c3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:29.571745 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 596/1184] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:15:29 2022] GET /identity/v3/projects/4a2443ef903c492b90316bf4786d5f5d => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:29.578482 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b4e8e-84b5-4440-8b31-6f0585c7d0ba None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:29.676922 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b4e8e-84b5-4440-8b31-6f0585c7d0ba None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:29.684145 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:29.711871 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f660ab-f56b-45ba-af86-893dd9a45b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:29.712643 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f660ab-f56b-45ba-af86-893dd9a45b29 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:29.713028 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f660ab-f56b-45ba-af86-893dd9a45b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:29.713392 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f660ab-f56b-45ba-af86-893dd9a45b29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:29.714311 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f660ab-f56b-45ba-af86-893dd9a45b29 None admin] RBAC: Adding query filter params (role.id=8acc34a163874a60b03728b814eaf075, scope.project.id=4a2443ef903c492b90316bf4786d5f5d, user.id=2fbd9323d27a473ca52ba437aec2aefa) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:15:29.714724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f660ab-f56b-45ba-af86-893dd9a45b29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:29.716278 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f660ab-f56b-45ba-af86-893dd9a45b29 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:29.721919 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 589/1185] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:15:29 2022] GET /identity/v3/role_assignments?user.id=2fbd9323d27a473ca52ba437aec2aefa&scope.project.id=4a2443ef903c492b90316bf4786d5f5d&role.id=8acc34a163874a60b03728b814eaf075 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:31.760934 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4b487-8094-4a6d-911c-e27fce22779a None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:31.761471 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4b487-8094-4a6d-911c-e27fce22779a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:31.761471 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4b487-8094-4a6d-911c-e27fce22779a None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:31.762990 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 597/1186] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:15:31.769153 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a6adc-f241-4af8-92d2-2fafc289c961 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:31.769814 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a6adc-f241-4af8-92d2-2fafc289c961 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:31.770007 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a6adc-f241-4af8-92d2-2fafc289c961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:31.786636 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-333a6adc-f241-4af8-92d2-2fafc289c961 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:15:31.787868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-333a6adc-f241-4af8-92d2-2fafc289c961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:15:31.853701 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 590/1187] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:15:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:15:31.877594 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4b487-8094-4a6d-911c-e27fce22779a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:31.886840 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:31.981534 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4b487-8094-4a6d-911c-e27fce22779a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:32.016406 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d2524-87b4-49e1-82b3-b431fd368af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:32.017123 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d2524-87b4-49e1-82b3-b431fd368af1 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:32.017404 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d2524-87b4-49e1-82b3-b431fd368af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:32.017677 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d2524-87b4-49e1-82b3-b431fd368af1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:32.027997 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d2524-87b4-49e1-82b3-b431fd368af1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:15:32.035873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-f60d2524-87b4-49e1-82b3-b431fd368af1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:15:32.035873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:15:32.035873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:15:32.035873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:15:32.035873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:15:32.035873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:15:32.035873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:15:32.035873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:15:32.035873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:15:32.035873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:15:32.035873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:15:32.035873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:15:32.035873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:15:32.035873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:15:32.035873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:15:32.035873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:15:32.036977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:15:32.036977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:15:32.036977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:15:32.036977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:15:32.036977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:15:32.036977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:15:32.036977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:15:32.036977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:15:32.036977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:15:32.036977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:15:32.036977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:15:32.036977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:15:32.036977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:15:32.036977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:15:32.036977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:15:32.036977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:15:32.037850 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:15:32.037850 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:15:32.037850 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:15:32.037850 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:15:32.037850 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:15:32.037850 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:15:32.037850 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:15:32.037850 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:15:32.037850 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:15:32.037850 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:15:32.037850 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:15:32.037850 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:15:32.037850 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 598/1188] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:15:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:15:32.043347 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a6adc-f241-4af8-92d2-2fafc289c961 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:32.138664 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:32.141590 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a6adc-f241-4af8-92d2-2fafc289c961 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:32.175096 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffb417-4939-4471-9d43-a0e727193b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:32.175778 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffb417-4939-4471-9d43-a0e727193b78 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:32.176065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffb417-4939-4471-9d43-a0e727193b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:32.176345 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffb417-4939-4471-9d43-a0e727193b78 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:32.177080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffb417-4939-4471-9d43-a0e727193b78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:15:32.177388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffb417-4939-4471-9d43-a0e727193b78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:32.178755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffb417-4939-4471-9d43-a0e727193b78 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:32.187751 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 591/1189] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:15:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:32.193700 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d2524-87b4-49e1-82b3-b431fd368af1 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:32.261435 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:32.297289 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d2524-87b4-49e1-82b3-b431fd368af1 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:32.330926 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c139e60-f222-4517-8655-5512028acd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:32.331818 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c139e60-f222-4517-8655-5512028acd72 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:32.332103 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c139e60-f222-4517-8655-5512028acd72 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:32.332382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c139e60-f222-4517-8655-5512028acd72 None admin] PATH_INFO: `/v3/users/2fbd9323d27a473ca52ba437aec2aefa` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:32.333561 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c139e60-f222-4517-8655-5512028acd72 None admin] RBAC: Authorizing `identity:get_user(user_id=2fbd9323d27a473ca52ba437aec2aefa)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:32.335012 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c139e60-f222-4517-8655-5512028acd72 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:32.336687 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 599/1190] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:15:32 2022] GET /identity/v3/users/2fbd9323d27a473ca52ba437aec2aefa => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:32.343272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffb417-4939-4471-9d43-a0e727193b78 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:32.443027 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffb417-4939-4471-9d43-a0e727193b78 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:32.475855 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a2fb0-e750-4608-9d54-eb5ca92cc2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:32.476716 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a2fb0-e750-4608-9d54-eb5ca92cc2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:32.477001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a2fb0-e750-4608-9d54-eb5ca92cc2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:32.477276 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a2fb0-e750-4608-9d54-eb5ca92cc2b5 None admin] PATH_INFO: `/v3/projects/4a2443ef903c492b90316bf4786d5f5d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:32.478446 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a2fb0-e750-4608-9d54-eb5ca92cc2b5 None admin] RBAC: Authorizing `identity:get_project(project_id=4a2443ef903c492b90316bf4786d5f5d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:32.479907 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a2fb0-e750-4608-9d54-eb5ca92cc2b5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:32.481598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 592/1191] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:15:32 2022] GET /identity/v3/projects/4a2443ef903c492b90316bf4786d5f5d => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:15:32.487688 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c139e60-f222-4517-8655-5512028acd72 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:32.587301 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c139e60-f222-4517-8655-5512028acd72 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:32.619698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8866688-05a7-4017-9e92-2c2814e47e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:32.620578 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8866688-05a7-4017-9e92-2c2814e47e9a None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:32.620865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8866688-05a7-4017-9e92-2c2814e47e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:32.621147 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8866688-05a7-4017-9e92-2c2814e47e9a None admin] PATH_INFO: `/v3/projects/4a2443ef903c492b90316bf4786d5f5d/users/2fbd9323d27a473ca52ba437aec2aefa/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:32.623103 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8866688-05a7-4017-9e92-2c2814e47e9a None admin] RBAC: Authorizing `identity:create_grant(project_id=4a2443ef903c492b90316bf4786d5f5d, user_id=2fbd9323d27a473ca52ba437aec2aefa, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:32.625262 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8866688-05a7-4017-9e92-2c2814e47e9a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:32.627446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:32.696939 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 600/1192] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 30 16:15:32 2022] PUT /identity/v3/projects/4a2443ef903c492b90316bf4786d5f5d/users/2fbd9323d27a473ca52ba437aec2aefa/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:15:34.731518 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d8f92-3f5c-4ef9-b052-7fc8704be385 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:34.732092 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d8f92-3f5c-4ef9-b052-7fc8704be385 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:34.732092 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d8f92-3f5c-4ef9-b052-7fc8704be385 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:34.733489 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 593/1193] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:15:34.738684 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50bf6b4-4c49-4ff5-a03f-ec08682bee37 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:34.739081 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50bf6b4-4c49-4ff5-a03f-ec08682bee37 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:34.739251 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50bf6b4-4c49-4ff5-a03f-ec08682bee37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:34.756229 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f50bf6b4-4c49-4ff5-a03f-ec08682bee37 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:15:34.757412 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f50bf6b4-4c49-4ff5-a03f-ec08682bee37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:15:34.822291 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 601/1194] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:15:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:15:34.847266 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d8f92-3f5c-4ef9-b052-7fc8704be385 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:34.878075 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:34.957408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d8f92-3f5c-4ef9-b052-7fc8704be385 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:34.990671 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc11d82-6105-48a8-85cf-d2a77f23768b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:34.991415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc11d82-6105-48a8-85cf-d2a77f23768b None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:34.991692 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc11d82-6105-48a8-85cf-d2a77f23768b None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:34.991973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc11d82-6105-48a8-85cf-d2a77f23768b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:35.002854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc11d82-6105-48a8-85cf-d2a77f23768b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:15:35.010616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-4bc11d82-6105-48a8-85cf-d2a77f23768b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:15:35.010616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:15:35.010616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:15:35.010616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:15:35.010616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:15:35.010616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:15:35.010616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:15:35.010616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:15:35.010616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:15:35.010616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:15:35.010616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:15:35.010616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:15:35.010616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:15:35.010616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:15:35.010616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:15:35.010616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:15:35.011810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:15:35.011810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:15:35.011810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:15:35.011810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:15:35.011810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:15:35.011810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:15:35.011810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:15:35.011810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:15:35.011810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:15:35.011810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:15:35.011810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:15:35.011810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:15:35.011810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:15:35.011810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:15:35.011810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:15:35.011810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:15:35.012927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:15:35.012927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:15:35.012927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:15:35.012927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:15:35.012927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:15:35.012927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:15:35.012927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:15:35.012927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:15:35.012927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:15:35.012927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:15:35.012927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:15:35.012927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:15:35.012927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 594/1195] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:15:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:15:35.018094 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50bf6b4-4c49-4ff5-a03f-ec08682bee37 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:35.117298 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50bf6b4-4c49-4ff5-a03f-ec08682bee37 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:35.141045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:35.151412 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025cca3-3f1b-486c-82f8-a520f89ae511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:35.152088 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025cca3-3f1b-486c-82f8-a520f89ae511 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:35.152443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025cca3-3f1b-486c-82f8-a520f89ae511 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:35.152625 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8025cca3-3f1b-486c-82f8-a520f89ae511 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:35.153355 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8025cca3-3f1b-486c-82f8-a520f89ae511 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:15:35.153655 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8025cca3-3f1b-486c-82f8-a520f89ae511 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:35.155059 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8025cca3-3f1b-486c-82f8-a520f89ae511 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:35.163346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 602/1196] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:15:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:35.169271 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc11d82-6105-48a8-85cf-d2a77f23768b None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:35.265427 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:35.275054 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc11d82-6105-48a8-85cf-d2a77f23768b None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:35.308342 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74442165-d61b-4b16-9eb5-1bcb63f11fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:35.309193 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74442165-d61b-4b16-9eb5-1bcb63f11fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:35.309475 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74442165-d61b-4b16-9eb5-1bcb63f11fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:35.309749 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74442165-d61b-4b16-9eb5-1bcb63f11fd9 None admin] PATH_INFO: `/v3/users/2fbd9323d27a473ca52ba437aec2aefa` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:35.310921 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74442165-d61b-4b16-9eb5-1bcb63f11fd9 None admin] RBAC: Authorizing `identity:get_user(user_id=2fbd9323d27a473ca52ba437aec2aefa)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:35.313107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74442165-d61b-4b16-9eb5-1bcb63f11fd9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:35.315053 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 595/1197] 10.222.0.67 () {56 vars in 1177 bytes} [Thu Jun 30 16:15:35 2022] GET /identity/v3/users/2fbd9323d27a473ca52ba437aec2aefa => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:35.320960 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025cca3-3f1b-486c-82f8-a520f89ae511 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:35.419452 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8025cca3-3f1b-486c-82f8-a520f89ae511 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:35.453116 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69550307-8bd7-4b73-86d5-c0233e3a3815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:35.453757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69550307-8bd7-4b73-86d5-c0233e3a3815 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:35.455904 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69550307-8bd7-4b73-86d5-c0233e3a3815 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:35.455904 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69550307-8bd7-4b73-86d5-c0233e3a3815 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:35.509064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:35.522759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69550307-8bd7-4b73-86d5-c0233e3a3815 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:35.524198 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69550307-8bd7-4b73-86d5-c0233e3a3815 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:35.629223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 603/1198] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:15:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 16:15:35.635085 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74442165-d61b-4b16-9eb5-1bcb63f11fd9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:35.735363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74442165-d61b-4b16-9eb5-1bcb63f11fd9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:35.769109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149d576-ebba-4d3a-83c4-60490e441b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:35.770206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149d576-ebba-4d3a-83c4-60490e441b64 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:35.770485 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149d576-ebba-4d3a-83c4-60490e441b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:35.770756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149d576-ebba-4d3a-83c4-60490e441b64 None admin] PATH_INFO: `/v3/projects/4a2443ef903c492b90316bf4786d5f5d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:35.771939 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5149d576-ebba-4d3a-83c4-60490e441b64 None admin] RBAC: Authorizing `identity:get_project(project_id=4a2443ef903c492b90316bf4786d5f5d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:35.773393 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5149d576-ebba-4d3a-83c4-60490e441b64 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:35.775027 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 596/1199] 10.222.0.67 () {56 vars in 1186 bytes} [Thu Jun 30 16:15:35 2022] GET /identity/v3/projects/4a2443ef903c492b90316bf4786d5f5d => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:35.781145 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69550307-8bd7-4b73-86d5-c0233e3a3815 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:15:35.864387 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:35.885632 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69550307-8bd7-4b73-86d5-c0233e3a3815 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:15:35.920411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e909b5c-083d-4035-b52c-7e77ea714412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:15:35.921582 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e909b5c-083d-4035-b52c-7e77ea714412 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:15:35.921850 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e909b5c-083d-4035-b52c-7e77ea714412 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:15:35.922127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e909b5c-083d-4035-b52c-7e77ea714412 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:15:35.922997 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e909b5c-083d-4035-b52c-7e77ea714412 None admin] RBAC: Adding query filter params (role.id=8acc34a163874a60b03728b814eaf075, scope.project.id=4a2443ef903c492b90316bf4786d5f5d, user.id=2fbd9323d27a473ca52ba437aec2aefa) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:15:35.923301 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e909b5c-083d-4035-b52c-7e77ea714412 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:15:35.924763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e909b5c-083d-4035-b52c-7e77ea714412 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:15:35.930519 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 604/1200] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:15:35 2022] GET /identity/v3/role_assignments?user.id=2fbd9323d27a473ca52ba437aec2aefa&scope.project.id=4a2443ef903c492b90316bf4786d5f5d&role.id=8acc34a163874a60b03728b814eaf075 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:15:45.764469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=87 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:15:45.927891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=25 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:16:57.728043 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade8509-1534-47ee-b30c-cd000e9f2c87 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:16:57.728626 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade8509-1534-47ee-b30c-cd000e9f2c87 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:16:57.728626 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ade8509-1534-47ee-b30c-cd000e9f2c87 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:16:57.730044 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 597/1201] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:16:57.735952 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94101e8-63e1-4e29-a489-ff99b61076a7 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:16:57.736408 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94101e8-63e1-4e29-a489-ff99b61076a7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:16:57.736519 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94101e8-63e1-4e29-a489-ff99b61076a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:16:57.753725 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e94101e8-63e1-4e29-a489-ff99b61076a7 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:16:57.758353 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e94101e8-63e1-4e29-a489-ff99b61076a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:16:57.822676 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 605/1202] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:16:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:16:57.849253 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade8509-1534-47ee-b30c-cd000e9f2c87 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:16:57.952620 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ade8509-1534-47ee-b30c-cd000e9f2c87 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:16:57.987563 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb253f8a-f533-43b1-9199-9c4fea830c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:16:57.988282 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb253f8a-f533-43b1-9199-9c4fea830c65 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:16:57.988560 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb253f8a-f533-43b1-9199-9c4fea830c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:16:57.988833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb253f8a-f533-43b1-9199-9c4fea830c65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:16:58.011706 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb253f8a-f533-43b1-9199-9c4fea830c65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:16:58.011706 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb253f8a-f533-43b1-9199-9c4fea830c65 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:16:58.020333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-eb253f8a-f533-43b1-9199-9c4fea830c65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:16:58.020333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:16:58.020333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:16:58.020333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:16:58.020333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:16:58.020333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:16:58.020333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:16:58.020333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:16:58.020333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:16:58.020333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:16:58.020333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:16:58.020333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:16:58.020333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:16:58.020333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:16:58.020333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:16:58.021514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:16:58.021514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:16:58.021514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:16:58.021514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:16:58.021514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:16:58.021514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:16:58.021514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:16:58.021514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:16:58.021514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:16:58.021514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:16:58.021514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:16:58.021514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:16:58.021514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:16:58.021514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:16:58.021514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:16:58.021514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:16:58.021514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:16:58.022743 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:16:58.022743 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:16:58.022743 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:16:58.022743 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:16:58.022743 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 598/1203] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:16:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:16:58.027522 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94101e8-63e1-4e29-a489-ff99b61076a7 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:16:58.126814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94101e8-63e1-4e29-a489-ff99b61076a7 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:16:58.155554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:16:58.161106 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89017448-8e51-494a-b190-e7cc8bb79091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:16:58.161755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89017448-8e51-494a-b190-e7cc8bb79091 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:16:58.162034 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89017448-8e51-494a-b190-e7cc8bb79091 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:16:58.162307 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89017448-8e51-494a-b190-e7cc8bb79091 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:16:58.163014 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89017448-8e51-494a-b190-e7cc8bb79091 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:16:58.163317 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89017448-8e51-494a-b190-e7cc8bb79091 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:16:58.164721 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89017448-8e51-494a-b190-e7cc8bb79091 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:16:58.176580 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 606/1204] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:16:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:16:58.182399 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb253f8a-f533-43b1-9199-9c4fea830c65 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:16:58.237099 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:16:58.283550 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb253f8a-f533-43b1-9199-9c4fea830c65 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:16:58.317602 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a132582-dfde-4a3e-8114-85d8a17cabe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:16:58.318317 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a132582-dfde-4a3e-8114-85d8a17cabe7 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:16:58.318602 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a132582-dfde-4a3e-8114-85d8a17cabe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:16:58.318876 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a132582-dfde-4a3e-8114-85d8a17cabe7 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:16:58.319611 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a132582-dfde-4a3e-8114-85d8a17cabe7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:16:58.321055 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a132582-dfde-4a3e-8114-85d8a17cabe7 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:16:58.469080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 599/1205] 10.222.0.67 () {60 vars in 1132 bytes} [Thu Jun 30 16:16:58 2022] POST /identity/v3/users => generated 251 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:17:00.559056 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9fe01-af6b-482e-a682-afc1ff78e0e5 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:00.559629 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9fe01-af6b-482e-a682-afc1ff78e0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:00.559629 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9fe01-af6b-482e-a682-afc1ff78e0e5 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:00.562065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 607/1206] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:17:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:17:00.567360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d58855-40c7-4985-84b5-007f4a0312b6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:00.567875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d58855-40c7-4985-84b5-007f4a0312b6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:00.567988 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d58855-40c7-4985-84b5-007f4a0312b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:00.584652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c2d58855-40c7-4985-84b5-007f4a0312b6 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:17:00.585847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c2d58855-40c7-4985-84b5-007f4a0312b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:17:00.651555 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 600/1207] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:17:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:17:00.676945 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9fe01-af6b-482e-a682-afc1ff78e0e5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:00.779781 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9fe01-af6b-482e-a682-afc1ff78e0e5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:00.816160 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5317ea9-d25a-4735-a4d5-52c74f894504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:00.816866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5317ea9-d25a-4735-a4d5-52c74f894504 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:00.817148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5317ea9-d25a-4735-a4d5-52c74f894504 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:00.817423 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5317ea9-d25a-4735-a4d5-52c74f894504 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:00.827723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5317ea9-d25a-4735-a4d5-52c74f894504 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:17:00.835348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-c5317ea9-d25a-4735-a4d5-52c74f894504 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:17:00.835348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:00.835348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:00.835348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:00.835348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:00.835348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:00.835348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:00.835348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:00.835348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:00.835348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:00.835348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:00.835348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:00.835348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:17:00.835348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:17:00.835348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:17:00.835348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:17:00.836482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:17:00.836482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:17:00.836482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:17:00.836482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:17:00.836482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:00.836482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:00.836482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:00.836482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:00.836482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:00.836482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:00.836482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:00.836482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:00.836482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:00.836482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:00.836482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:00.836482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:00.837710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:00.837710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:00.837710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:00.837710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:00.837710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:17:00.837710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:17:00.837710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:17:00.837710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:17:00.837710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:17:00.837710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:17:00.837710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:17:00.837710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:17:00.837710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 608/1208] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 30 16:17:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:00.843560 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d58855-40c7-4985-84b5-007f4a0312b6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:00.902915 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:00.964198 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d58855-40c7-4985-84b5-007f4a0312b6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:00.999269 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b9c9d-ae38-482a-bc37-f34d829b50a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:01.000007 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b9c9d-ae38-482a-bc37-f34d829b50a0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:01.000264 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b9c9d-ae38-482a-bc37-f34d829b50a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:01.000537 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b9c9d-ae38-482a-bc37-f34d829b50a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:01.001277 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b9c9d-ae38-482a-bc37-f34d829b50a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:01.001584 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b9c9d-ae38-482a-bc37-f34d829b50a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:01.003000 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b9c9d-ae38-482a-bc37-f34d829b50a0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:01.011197 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 601/1209] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 30 16:17:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:01.017224 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5317ea9-d25a-4735-a4d5-52c74f894504 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:01.068620 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:01.118769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5317ea9-d25a-4735-a4d5-52c74f894504 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:01.152210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0da69-b6aa-4a5d-b721-a8296cc9f063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:01.152993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0da69-b6aa-4a5d-b721-a8296cc9f063 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:01.153278 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0da69-b6aa-4a5d-b721-a8296cc9f063 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:01.153552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0da69-b6aa-4a5d-b721-a8296cc9f063 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:01.164843 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e0da69-b6aa-4a5d-b721-a8296cc9f063 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:01.166299 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e0da69-b6aa-4a5d-b721-a8296cc9f063 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:01.183576 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-09e0da69-b6aa-4a5d-b721-a8296cc9f063 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:17:01.183576 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:01.183576 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:01.183576 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:01.183576 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:01.183576 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:01.183576 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:01.183576 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:01.183576 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:01.183576 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:01.183576 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:01.183576 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:01.183576 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:17:01.183576 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:17:01.183576 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:17:01.184804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:17:01.184804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:01.184804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:01.184804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:01.184804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:01.184804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:01.184804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:01.184804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:01.184804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:01.184804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:01.184804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:01.184804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:01.184804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:01.184804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:01.184804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:01.184804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:01.184804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:01.186436 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:17:01.186436 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:17:01.186436 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:17:01.186436 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:17:01.186436 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 609/1210] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:17:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:01.190736 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b9c9d-ae38-482a-bc37-f34d829b50a0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:01.297037 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b9c9d-ae38-482a-bc37-f34d829b50a0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:01.330226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ea9db-ee20-426e-b677-a7287a802ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:01.330814 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ea9db-ee20-426e-b677-a7287a802ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:01.331096 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ea9db-ee20-426e-b677-a7287a802ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:01.331371 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ea9db-ee20-426e-b677-a7287a802ef1 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:01.332114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ea9db-ee20-426e-b677-a7287a802ef1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:01.332415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ea9db-ee20-426e-b677-a7287a802ef1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:01.333776 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ea9db-ee20-426e-b677-a7287a802ef1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:01.345977 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 602/1211] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:01.351752 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:01.351980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0da69-b6aa-4a5d-b721-a8296cc9f063 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:01.453790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0da69-b6aa-4a5d-b721-a8296cc9f063 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:01.487857 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07e4f1-6cc4-46b3-901a-d64e920a836d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:01.488926 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07e4f1-6cc4-46b3-901a-d64e920a836d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:01.490248 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:01.490694 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07e4f1-6cc4-46b3-901a-d64e920a836d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:01.491021 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07e4f1-6cc4-46b3-901a-d64e920a836d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:01.499163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07e4f1-6cc4-46b3-901a-d64e920a836d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:01.500658 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07e4f1-6cc4-46b3-901a-d64e920a836d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:01.516338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-8d07e4f1-6cc4-46b3-901a-d64e920a836d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 16:17:01.516338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:01.516338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:01.516338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:01.516338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:01.516338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:01.516338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:01.516338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:01.516338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:01.516338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:01.516338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:01.516338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:01.516338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:17:01.516338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:17:01.516338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:17:01.516338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:17:01.517517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:01.517517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:01.517517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:17:01.517517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:17:01.517517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:17:01.517517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:17:01.517517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:01.517517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:01.517517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:01.517517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:01.517517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:01.517517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:01.517517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:01.517517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:01.517517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:01.517517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:01.517517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:01.517517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:01.518703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:01.518703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:01.518703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:17:01.518703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:17:01.518703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:17:01.518703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:17:01.518703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:17:01.518703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:17:01.518703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 16:17:01.518703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:17:01.518703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 610/1212] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:17:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:01.523637 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ea9db-ee20-426e-b677-a7287a802ef1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:01.620746 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ea9db-ee20-426e-b677-a7287a802ef1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:01.656339 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb8640-e186-40ae-aeb1-2e417f694365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:01.657001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb8640-e186-40ae-aeb1-2e417f694365 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:01.657281 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb8640-e186-40ae-aeb1-2e417f694365 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:01.657554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb8640-e186-40ae-aeb1-2e417f694365 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:01.658381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb8640-e186-40ae-aeb1-2e417f694365 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:01.658685 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb8640-e186-40ae-aeb1-2e417f694365 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:01.660150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb8640-e186-40ae-aeb1-2e417f694365 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:01.678722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 603/1213] 10.222.0.67 () {56 vars in 1143 bytes} [Thu Jun 30 16:17:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:01.684604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07e4f1-6cc4-46b3-901a-d64e920a836d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:01.786906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07e4f1-6cc4-46b3-901a-d64e920a836d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:01.820988 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b66fdd-7b39-451b-9471-cd0f3d34b713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:01.821650 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b66fdd-7b39-451b-9471-cd0f3d34b713 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:01.821931 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b66fdd-7b39-451b-9471-cd0f3d34b713 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:01.822203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b66fdd-7b39-451b-9471-cd0f3d34b713 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:01.854755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:01.893835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b66fdd-7b39-451b-9471-cd0f3d34b713 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:01.895233 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b66fdd-7b39-451b-9471-cd0f3d34b713 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:01.993453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 611/1214] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:17:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 16:17:01.999959 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb8640-e186-40ae-aeb1-2e417f694365 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:02.019540 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:02.101191 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb8640-e186-40ae-aeb1-2e417f694365 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:02.135450 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf616f8e-4f60-43df-b187-7771abcd2c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:02.136279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf616f8e-4f60-43df-b187-7771abcd2c45 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:02.136561 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf616f8e-4f60-43df-b187-7771abcd2c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:02.136835 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf616f8e-4f60-43df-b187-7771abcd2c45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:02.152201 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf616f8e-4f60-43df-b187-7771abcd2c45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:02.152201 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf616f8e-4f60-43df-b187-7771abcd2c45 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:02.166101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-bf616f8e-4f60-43df-b187-7771abcd2c45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:17:02.166101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:02.166101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:02.166101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:02.166101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:02.166101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:02.166101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:02.166101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:02.166101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:02.166101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:02.166101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:02.166101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:02.166101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:17:02.166101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:17:02.166101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:17:02.167226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:17:02.167226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:02.167226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:02.167226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:02.167226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:02.167226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:02.167226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:02.167226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:02.167226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:02.167226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:02.167226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:02.167226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:02.167226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:02.167226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:02.167226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:02.167226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:02.167226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:02.168364 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:17:02.168364 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:17:02.168364 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:17:02.168364 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:17:02.168364 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 604/1215] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:17:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:02.173729 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b66fdd-7b39-451b-9471-cd0f3d34b713 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:02.238382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:02.278215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b66fdd-7b39-451b-9471-cd0f3d34b713 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:02.312573 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de547c2d-60ac-4982-a056-d88e788dedf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:02.313245 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de547c2d-60ac-4982-a056-d88e788dedf2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:02.313532 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de547c2d-60ac-4982-a056-d88e788dedf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:02.313813 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de547c2d-60ac-4982-a056-d88e788dedf2 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:02.314530 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de547c2d-60ac-4982-a056-d88e788dedf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:02.314842 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de547c2d-60ac-4982-a056-d88e788dedf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:02.316311 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de547c2d-60ac-4982-a056-d88e788dedf2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:02.328087 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 612/1216] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:02.334111 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf616f8e-4f60-43df-b187-7771abcd2c45 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:02.406522 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:02.437739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf616f8e-4f60-43df-b187-7771abcd2c45 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:02.471413 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96832486-ef9e-44ce-99e3-227dfc7a201c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:02.472286 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96832486-ef9e-44ce-99e3-227dfc7a201c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:02.472569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96832486-ef9e-44ce-99e3-227dfc7a201c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:02.472847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96832486-ef9e-44ce-99e3-227dfc7a201c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:02.479526 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96832486-ef9e-44ce-99e3-227dfc7a201c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:02.481001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96832486-ef9e-44ce-99e3-227dfc7a201c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:02.495575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-96832486-ef9e-44ce-99e3-227dfc7a201c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:17:02.495575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:02.495575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:02.495575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:02.495575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:02.495575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:02.495575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:02.495575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:02.495575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:02.495575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:02.495575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:02.495575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:02.495575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:17:02.495575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:17:02.495575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:17:02.496629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:17:02.496629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:02.496629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:02.496629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:02.496629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:02.496629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:02.496629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:02.496629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:02.496629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:02.496629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:02.496629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:02.496629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:02.496629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:02.496629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:02.496629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:02.496629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:02.496629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:02.497662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:17:02.497662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:17:02.497662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:17:02.497662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:17:02.497662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:17:02.497662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:17:02.497662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:17:02.497662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:17:02.497662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 605/1217] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:17:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:02.502674 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de547c2d-60ac-4982-a056-d88e788dedf2 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:02.604933 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de547c2d-60ac-4982-a056-d88e788dedf2 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:02.627973 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:02.640120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5df8c-e106-4584-a513-177a9d98c32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:02.641047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5df8c-e106-4584-a513-177a9d98c32e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:02.642073 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5df8c-e106-4584-a513-177a9d98c32e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:02.642073 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5df8c-e106-4584-a513-177a9d98c32e None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:02.642431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac5df8c-e106-4584-a513-177a9d98c32e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:02.642745 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac5df8c-e106-4584-a513-177a9d98c32e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:02.644335 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac5df8c-e106-4584-a513-177a9d98c32e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:02.656473 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 613/1218] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:17:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:02.662922 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96832486-ef9e-44ce-99e3-227dfc7a201c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:02.766518 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96832486-ef9e-44ce-99e3-227dfc7a201c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:02.793745 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:02.800173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56696977-974f-4147-aefe-eff0020815ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:02.800823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56696977-974f-4147-aefe-eff0020815ad None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:02.801107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56696977-974f-4147-aefe-eff0020815ad None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:02.801380 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56696977-974f-4147-aefe-eff0020815ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:02.802199 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56696977-974f-4147-aefe-eff0020815ad None admin] RBAC: Adding query filter params (role.id=57bc5df632b3407d97116ae34d92a5cb, scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d, user.id=a932a9390fdd47f3adcf053a60250944) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:02.802508 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56696977-974f-4147-aefe-eff0020815ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:02.803972 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56696977-974f-4147-aefe-eff0020815ad None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:02.809381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 606/1219] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:17:02 2022] GET /identity/v3/role_assignments?user.id=a932a9390fdd47f3adcf053a60250944&scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d&role.id=57bc5df632b3407d97116ae34d92a5cb => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:17:04.859446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015be66b-4c8c-43e7-a3dd-06511480f188 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:04.860030 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015be66b-4c8c-43e7-a3dd-06511480f188 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:04.860030 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015be66b-4c8c-43e7-a3dd-06511480f188 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:04.861452 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 614/1220] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:17:04.866768 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ad6b1-6b2f-4118-827a-9d399e3fe47b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:04.867181 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ad6b1-6b2f-4118-827a-9d399e3fe47b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:04.867328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ad6b1-6b2f-4118-827a-9d399e3fe47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:04.884413 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4a2ad6b1-6b2f-4118-827a-9d399e3fe47b None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:17:04.885604 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4a2ad6b1-6b2f-4118-827a-9d399e3fe47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:17:04.952884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 607/1221] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:17:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:17:04.975542 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015be66b-4c8c-43e7-a3dd-06511480f188 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:05.083898 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015be66b-4c8c-43e7-a3dd-06511480f188 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:05.118144 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75137fee-0ea2-4800-b799-d86d4c9222ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:05.118870 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75137fee-0ea2-4800-b799-d86d4c9222ab None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:05.119154 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75137fee-0ea2-4800-b799-d86d4c9222ab None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:05.119433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75137fee-0ea2-4800-b799-d86d4c9222ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:05.130150 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75137fee-0ea2-4800-b799-d86d4c9222ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:17:05.138239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-75137fee-0ea2-4800-b799-d86d4c9222ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:17:05.138239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:05.138239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:05.138239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:05.138239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:05.138239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:05.138239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:05.138239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:05.138239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:05.138239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:05.138239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:05.138239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:05.138239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:17:05.138239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:17:05.138239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:17:05.138239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:17:05.139373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:17:05.139373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:17:05.139373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:17:05.139373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:17:05.139373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:05.139373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:05.139373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:05.139373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:05.139373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:05.139373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:05.139373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:05.139373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:05.139373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:05.139373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:05.139373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:05.139373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:05.140863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:05.140863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:05.140863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:05.140863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:05.140863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:17:05.140863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:17:05.140863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:17:05.140863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:17:05.140863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:17:05.140863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:17:05.140863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:17:05.140863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:17:05.140863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 615/1222] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 30 16:17:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:05.145997 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ad6b1-6b2f-4118-827a-9d399e3fe47b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:05.248849 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ad6b1-6b2f-4118-827a-9d399e3fe47b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:05.272786 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:05.284169 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcdde0a-ec04-425f-8e00-e3e3b55dd3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:05.284831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcdde0a-ec04-425f-8e00-e3e3b55dd3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:05.285108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcdde0a-ec04-425f-8e00-e3e3b55dd3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:05.285378 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcdde0a-ec04-425f-8e00-e3e3b55dd3c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:05.286109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcdde0a-ec04-425f-8e00-e3e3b55dd3c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:05.286414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcdde0a-ec04-425f-8e00-e3e3b55dd3c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:05.287808 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcdde0a-ec04-425f-8e00-e3e3b55dd3c9 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:05.296634 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 608/1223] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 30 16:17:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:05.302581 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75137fee-0ea2-4800-b799-d86d4c9222ab None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:05.346174 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:05.403107 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75137fee-0ea2-4800-b799-d86d4c9222ab None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:05.437139 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fcd35-a191-42e5-994b-67ef04e65980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:05.437930 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fcd35-a191-42e5-994b-67ef04e65980 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:05.438212 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fcd35-a191-42e5-994b-67ef04e65980 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:05.438492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fcd35-a191-42e5-994b-67ef04e65980 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:05.449207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969fcd35-a191-42e5-994b-67ef04e65980 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:05.450708 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969fcd35-a191-42e5-994b-67ef04e65980 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:05.468216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-969fcd35-a191-42e5-994b-67ef04e65980 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:17:05.468216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:05.468216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:05.468216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:05.468216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:05.468216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:05.468216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:05.468216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:05.468216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:05.468216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:05.468216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:05.468216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:05.468216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:17:05.468216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:17:05.468216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:17:05.469163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:17:05.469163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:05.469163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:05.469163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:05.469163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:05.469163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:05.469163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:05.469163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:05.469163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:05.469163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:05.469163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:05.469163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:05.469163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:05.469163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:05.469163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:05.469163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:05.469163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:05.470096 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:17:05.470096 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:17:05.470096 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:17:05.470096 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:17:05.470096 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 616/1224] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:17:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:05.475282 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcdde0a-ec04-425f-8e00-e3e3b55dd3c9 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:05.578019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcdde0a-ec04-425f-8e00-e3e3b55dd3c9 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:05.612749 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af909bf-24e7-4b05-830f-b6d79b0c247f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:05.613415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af909bf-24e7-4b05-830f-b6d79b0c247f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:05.613690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af909bf-24e7-4b05-830f-b6d79b0c247f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:05.614839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af909bf-24e7-4b05-830f-b6d79b0c247f None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:05.614839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af909bf-24e7-4b05-830f-b6d79b0c247f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:05.615097 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af909bf-24e7-4b05-830f-b6d79b0c247f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:05.616335 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af909bf-24e7-4b05-830f-b6d79b0c247f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:05.628537 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 609/1225] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:17:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:05.634644 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fcd35-a191-42e5-994b-67ef04e65980 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:05.729435 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:05.735997 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fcd35-a191-42e5-994b-67ef04e65980 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:05.770531 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0001e6c3-a777-4132-8bca-e6810075f3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:05.771337 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001e6c3-a777-4132-8bca-e6810075f3ab None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:05.771653 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001e6c3-a777-4132-8bca-e6810075f3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:05.771935 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001e6c3-a777-4132-8bca-e6810075f3ab None admin] PATH_INFO: `/v3/users/placement` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:05.779461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0001e6c3-a777-4132-8bca-e6810075f3ab None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:05.780942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0001e6c3-a777-4132-8bca-e6810075f3ab None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:05.795612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-0001e6c3-a777-4132-8bca-e6810075f3ab None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 16:17:05.795612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:05.795612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:05.795612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:05.795612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:05.795612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:05.795612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:05.795612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:05.795612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:05.795612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:05.795612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:05.795612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:05.795612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:17:05.795612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:17:05.795612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:17:05.795612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:17:05.796741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:05.796741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:05.796741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:17:05.796741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:17:05.796741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:17:05.796741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:17:05.796741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:05.796741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:05.796741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:05.796741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:05.796741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:05.796741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:05.796741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:05.796741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:05.796741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:05.796741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:05.796741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:05.796741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:05.798175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:05.798175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:05.798175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:17:05.798175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:17:05.798175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:17:05.798175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:17:05.798175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:17:05.798175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:17:05.798175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 16:17:05.798175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:17:05.798175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 617/1226] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:17:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:05.802919 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af909bf-24e7-4b05-830f-b6d79b0c247f None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:05.850422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:05.903366 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af909bf-24e7-4b05-830f-b6d79b0c247f None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:05.937438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fcfea4-ea14-42f9-974b-a04e165ad9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:05.938097 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fcfea4-ea14-42f9-974b-a04e165ad9e0 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:05.938379 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fcfea4-ea14-42f9-974b-a04e165ad9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:05.938650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fcfea4-ea14-42f9-974b-a04e165ad9e0 None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:05.939500 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fcfea4-ea14-42f9-974b-a04e165ad9e0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:05.939809 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fcfea4-ea14-42f9-974b-a04e165ad9e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:05.941225 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fcfea4-ea14-42f9-974b-a04e165ad9e0 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:05.958835 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 610/1227] 10.222.0.67 () {56 vars in 1143 bytes} [Thu Jun 30 16:17:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:05.964658 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0001e6c3-a777-4132-8bca-e6810075f3ab None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:06.070154 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0001e6c3-a777-4132-8bca-e6810075f3ab None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:06.104565 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ad661-883f-495b-8917-dbbaee5fc636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:06.105356 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ad661-883f-495b-8917-dbbaee5fc636 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:06.105641 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ad661-883f-495b-8917-dbbaee5fc636 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:06.105922 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ad661-883f-495b-8917-dbbaee5fc636 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:06.116004 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ad661-883f-495b-8917-dbbaee5fc636 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:06.117459 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ad661-883f-495b-8917-dbbaee5fc636 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:06.135140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-938ad661-883f-495b-8917-dbbaee5fc636 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:17:06.135140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:06.135140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:06.135140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:06.135140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:06.135140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:06.135140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:06.135140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:06.135140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:06.135140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:06.135140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:06.135140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:06.135140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:17:06.135140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:17:06.135140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:17:06.139588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:17:06.139588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:06.139588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:06.139588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:06.139588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:06.139588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:06.139588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:06.139588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:06.139588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:06.139588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:06.139588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:06.139588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:06.139588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:06.139588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:06.139588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:06.139588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:06.139588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:06.141067 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:17:06.141067 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:17:06.141067 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:17:06.141067 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:17:06.141067 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 618/1228] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:17:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:06.143704 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fcfea4-ea14-42f9-974b-a04e165ad9e0 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:06.246288 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fcfea4-ea14-42f9-974b-a04e165ad9e0 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:06.249760 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:06.283924 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23710e2a-d5d6-49bc-920b-6aaa5232bc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:06.284610 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23710e2a-d5d6-49bc-920b-6aaa5232bc68 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:06.285391 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23710e2a-d5d6-49bc-920b-6aaa5232bc68 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:06.285391 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23710e2a-d5d6-49bc-920b-6aaa5232bc68 None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:06.285878 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23710e2a-d5d6-49bc-920b-6aaa5232bc68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:06.286185 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23710e2a-d5d6-49bc-920b-6aaa5232bc68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:06.287596 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23710e2a-d5d6-49bc-920b-6aaa5232bc68 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:06.299613 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 611/1229] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:06.305512 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ad661-883f-495b-8917-dbbaee5fc636 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:06.322489 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:06.414900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ad661-883f-495b-8917-dbbaee5fc636 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:06.449807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6facb35-a3cf-463a-a887-5ff6b4eb1c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:06.451280 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6facb35-a3cf-463a-a887-5ff6b4eb1c43 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:06.451280 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6facb35-a3cf-463a-a887-5ff6b4eb1c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:06.451280 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6facb35-a3cf-463a-a887-5ff6b4eb1c43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:06.457964 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6facb35-a3cf-463a-a887-5ff6b4eb1c43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:06.459408 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6facb35-a3cf-463a-a887-5ff6b4eb1c43 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:06.473458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-c6facb35-a3cf-463a-a887-5ff6b4eb1c43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:17:06.473458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:06.473458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:06.473458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:06.473458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:06.473458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:06.473458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:06.473458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:06.473458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:06.473458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:06.473458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:06.473458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:06.473458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:17:06.473458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:17:06.473458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:17:06.474496 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:17:06.474496 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:06.474496 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:06.474496 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:06.474496 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:06.474496 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:06.474496 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:06.474496 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:06.474496 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:06.474496 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:06.474496 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:06.474496 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:06.474496 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:06.474496 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:06.474496 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:06.474496 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:06.474496 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:06.475449 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:17:06.475449 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:17:06.475449 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:17:06.475449 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:17:06.475449 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:17:06.475449 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:17:06.475449 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:17:06.475449 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:17:06.475449 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 619/1230] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:17:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:06.480834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23710e2a-d5d6-49bc-920b-6aaa5232bc68 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:06.583956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23710e2a-d5d6-49bc-920b-6aaa5232bc68 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:06.625999 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b496b33-fefe-4376-add7-0d6c9a2c642c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:06.626658 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b496b33-fefe-4376-add7-0d6c9a2c642c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:06.626940 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b496b33-fefe-4376-add7-0d6c9a2c642c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:06.627265 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b496b33-fefe-4376-add7-0d6c9a2c642c None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:06.628083 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b496b33-fefe-4376-add7-0d6c9a2c642c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:06.628391 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b496b33-fefe-4376-add7-0d6c9a2c642c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:06.629811 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b496b33-fefe-4376-add7-0d6c9a2c642c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:06.642238 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 612/1231] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:17:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:06.648250 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6facb35-a3cf-463a-a887-5ff6b4eb1c43 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:06.717786 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:06.750743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6facb35-a3cf-463a-a887-5ff6b4eb1c43 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:06.785395 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b845f-910d-4719-9542-c065f5829672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:06.786322 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b845f-910d-4719-9542-c065f5829672 None admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:06.786609 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b845f-910d-4719-9542-c065f5829672 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:06.786893 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b845f-910d-4719-9542-c065f5829672 None admin] PATH_INFO: `/v3/projects/19ef92ebcce94b90bf39ad3a58772a1d/users/a932a9390fdd47f3adcf053a60250944/roles/57bc5df632b3407d97116ae34d92a5cb` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:06.801175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b845f-910d-4719-9542-c065f5829672 None admin] RBAC: Authorizing `identity:create_grant(project_id=19ef92ebcce94b90bf39ad3a58772a1d, user_id=a932a9390fdd47f3adcf053a60250944, role_id=57bc5df632b3407d97116ae34d92a5cb)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:06.802620 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b845f-910d-4719-9542-c065f5829672 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:06.962378 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 620/1232] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 30 16:17:06 2022] PUT /identity/v3/projects/19ef92ebcce94b90bf39ad3a58772a1d/users/a932a9390fdd47f3adcf053a60250944/roles/57bc5df632b3407d97116ae34d92a5cb => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:17:09.046031 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad000189-d648-43e0-86cb-b2bf84c78df4 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:09.046610 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad000189-d648-43e0-86cb-b2bf84c78df4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:09.046610 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad000189-d648-43e0-86cb-b2bf84c78df4 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:09.048075 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 613/1233] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:17:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:17:09.053251 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0196d0-7629-4def-b5a8-09b1e2f32b23 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:09.053684 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0196d0-7629-4def-b5a8-09b1e2f32b23 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:09.053811 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0196d0-7629-4def-b5a8-09b1e2f32b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:09.070173 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-cf0196d0-7629-4def-b5a8-09b1e2f32b23 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:17:09.071354 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cf0196d0-7629-4def-b5a8-09b1e2f32b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:17:09.137199 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 621/1234] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:17:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:17:09.163057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad000189-d648-43e0-86cb-b2bf84c78df4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:09.189906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:09.272928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad000189-d648-43e0-86cb-b2bf84c78df4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:09.308841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008224c3-9119-4582-a84e-6a0959776ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:09.311097 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008224c3-9119-4582-a84e-6a0959776ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:09.311097 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008224c3-9119-4582-a84e-6a0959776ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:09.311097 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008224c3-9119-4582-a84e-6a0959776ed4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:09.321146 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008224c3-9119-4582-a84e-6a0959776ed4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:17:09.330318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-008224c3-9119-4582-a84e-6a0959776ed4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:17:09.330318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:09.330318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:09.330318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:09.330318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:09.330318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:09.330318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:09.330318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:09.330318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:09.330318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:09.330318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:09.330318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:09.330318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:17:09.330318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:17:09.330318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:17:09.330318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:17:09.331541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:17:09.331541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:17:09.331541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:17:09.331541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:17:09.331541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:09.331541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:09.331541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:09.331541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:09.331541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:09.331541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:09.331541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:09.331541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:09.331541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:09.331541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:09.331541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:09.331541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:09.332583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:09.332583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:09.332583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:09.332583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:09.332583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:17:09.332583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:17:09.332583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:17:09.332583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:17:09.332583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:17:09.332583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:17:09.332583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 16:17:09.332583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:17:09.332583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 614/1235] 10.222.0.67 () {56 vars in 1102 bytes} [Thu Jun 30 16:17:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:09.337638 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0196d0-7629-4def-b5a8-09b1e2f32b23 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:09.384016 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:09.438642 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0196d0-7629-4def-b5a8-09b1e2f32b23 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:09.473887 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545a3aa-c7f5-43e6-a4d0-00bd2321571a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:09.474544 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545a3aa-c7f5-43e6-a4d0-00bd2321571a None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:09.474825 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545a3aa-c7f5-43e6-a4d0-00bd2321571a None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:09.475100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a545a3aa-c7f5-43e6-a4d0-00bd2321571a None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:09.475896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545a3aa-c7f5-43e6-a4d0-00bd2321571a None admin] RBAC: Adding query filter params (name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:09.476173 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545a3aa-c7f5-43e6-a4d0-00bd2321571a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:09.477585 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a545a3aa-c7f5-43e6-a4d0-00bd2321571a None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:09.486575 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 622/1236] 10.222.0.67 () {56 vars in 1103 bytes} [Thu Jun 30 16:17:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:09.491967 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008224c3-9119-4582-a84e-6a0959776ed4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:09.577801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:09.593904 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008224c3-9119-4582-a84e-6a0959776ed4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:09.627926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe5548-7ec0-42f5-a764-d9b13ca4a616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:09.628717 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe5548-7ec0-42f5-a764-d9b13ca4a616 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:09.629000 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe5548-7ec0-42f5-a764-d9b13ca4a616 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:09.629278 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe5548-7ec0-42f5-a764-d9b13ca4a616 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:09.639168 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabe5548-7ec0-42f5-a764-d9b13ca4a616 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:09.640891 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabe5548-7ec0-42f5-a764-d9b13ca4a616 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:09.658267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-dabe5548-7ec0-42f5-a764-d9b13ca4a616 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:17:09.658267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:09.658267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:09.658267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:09.658267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:09.658267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:09.658267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:09.658267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:09.658267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:09.658267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:09.658267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:09.658267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:09.658267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:17:09.658267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:17:09.658267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:17:09.659619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:17:09.659619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:09.659619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:09.659619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:09.659619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:09.659619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:09.659619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:09.659619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:09.659619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:09.659619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:09.659619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:09.659619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:09.659619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:09.659619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:09.659619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:09.659619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:09.659619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:09.660713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:17:09.660713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:17:09.660713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:17:09.660713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:17:09.660713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 615/1237] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:17:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:09.665560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545a3aa-c7f5-43e6-a4d0-00bd2321571a None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:09.767782 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a545a3aa-c7f5-43e6-a4d0-00bd2321571a None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:09.776693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:09.801504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0e62b-bc3a-416b-8e2d-42779bb5a4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:09.802182 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0e62b-bc3a-416b-8e2d-42779bb5a4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:09.802468 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0e62b-bc3a-416b-8e2d-42779bb5a4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:09.802741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c0e62b-bc3a-416b-8e2d-42779bb5a4b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:09.803456 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0e62b-bc3a-416b-8e2d-42779bb5a4b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:09.803796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0e62b-bc3a-416b-8e2d-42779bb5a4b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:09.805266 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c0e62b-bc3a-416b-8e2d-42779bb5a4b3 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:09.818851 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 623/1238] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:09.824646 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe5548-7ec0-42f5-a764-d9b13ca4a616 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:09.924949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe5548-7ec0-42f5-a764-d9b13ca4a616 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:09.963859 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8f957-cb69-4b91-ab5b-3efbafb38b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:09.964652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8f957-cb69-4b91-ab5b-3efbafb38b2c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:09.964930 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8f957-cb69-4b91-ab5b-3efbafb38b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:09.965200 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8f957-cb69-4b91-ab5b-3efbafb38b2c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:09.968173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:09.972707 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a8f957-cb69-4b91-ab5b-3efbafb38b2c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:09.974151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a8f957-cb69-4b91-ab5b-3efbafb38b2c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:09.989527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-e9a8f957-cb69-4b91-ab5b-3efbafb38b2c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 16:17:09.989527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:09.989527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:09.989527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:09.989527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:09.989527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:09.989527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:09.989527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:09.989527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:09.989527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:09.989527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:09.989527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:09.989527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:17:09.989527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:17:09.989527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:17:09.989527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:17:09.990817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:09.990817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:09.990817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:17:09.990817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:17:09.990817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:17:09.990817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:17:09.990817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:09.990817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:09.990817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:09.990817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:09.990817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:09.990817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:09.990817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:09.990817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:09.990817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:09.990817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:09.990817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:09.990817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:09.991812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:09.991812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:09.991812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:17:09.991812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:17:09.991812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:17:09.991812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:17:09.991812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:17:09.991812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:17:09.991812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 16:17:09.991812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:17:09.991812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 616/1239] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:17:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:09.996608 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0e62b-bc3a-416b-8e2d-42779bb5a4b3 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:10.100147 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c0e62b-bc3a-416b-8e2d-42779bb5a4b3 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:10.134712 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a945d55-661c-4b92-b9f8-165ded1911fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:10.135375 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a945d55-661c-4b92-b9f8-165ded1911fd None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:10.135685 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a945d55-661c-4b92-b9f8-165ded1911fd None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:10.135968 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a945d55-661c-4b92-b9f8-165ded1911fd None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:10.136797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a945d55-661c-4b92-b9f8-165ded1911fd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:10.137107 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a945d55-661c-4b92-b9f8-165ded1911fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:10.138798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a945d55-661c-4b92-b9f8-165ded1911fd None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:10.156495 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 624/1240] 10.222.0.67 () {56 vars in 1143 bytes} [Thu Jun 30 16:17:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:10.162518 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8f957-cb69-4b91-ab5b-3efbafb38b2c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:10.263681 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8f957-cb69-4b91-ab5b-3efbafb38b2c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:10.298184 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12822ae4-ed96-45ed-be54-a0edbc8af77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:10.298840 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12822ae4-ed96-45ed-be54-a0edbc8af77d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:10.299120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12822ae4-ed96-45ed-be54-a0edbc8af77d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:10.299421 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12822ae4-ed96-45ed-be54-a0edbc8af77d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:10.361349 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:10.367713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12822ae4-ed96-45ed-be54-a0edbc8af77d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:10.369152 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12822ae4-ed96-45ed-be54-a0edbc8af77d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:10.472914 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 617/1241] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:17:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 16:17:10.478130 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a945d55-661c-4b92-b9f8-165ded1911fd None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:10.507355 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:10.576060 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a945d55-661c-4b92-b9f8-165ded1911fd None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:10.609499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8b46d-103a-4090-88bc-9bd36657d8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:10.610278 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8b46d-103a-4090-88bc-9bd36657d8eb None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:10.610557 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8b46d-103a-4090-88bc-9bd36657d8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:10.610834 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8b46d-103a-4090-88bc-9bd36657d8eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:10.620430 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8b46d-103a-4090-88bc-9bd36657d8eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:10.621932 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8b46d-103a-4090-88bc-9bd36657d8eb None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:10.638753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-89a8b46d-103a-4090-88bc-9bd36657d8eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:17:10.638753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:10.638753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:10.638753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:10.638753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:10.638753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:10.638753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:10.638753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:10.638753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:10.638753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:10.638753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:10.638753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:10.638753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:17:10.638753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:17:10.638753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:17:10.641649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:17:10.641649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:10.641649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:10.641649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:10.641649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:10.641649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:10.641649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:10.641649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:10.641649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:10.641649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:10.641649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:10.641649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:10.641649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:10.641649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:10.641649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:10.641649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:10.641649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:10.642747 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:17:10.642747 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:17:10.642747 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:17:10.642747 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:17:10.642747 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 625/1242] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:17:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:10.646033 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12822ae4-ed96-45ed-be54-a0edbc8af77d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:10.745956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:10.750176 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12822ae4-ed96-45ed-be54-a0edbc8af77d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:10.786113 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d65480-37c7-43cf-bae6-f95222a63f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:10.786779 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d65480-37c7-43cf-bae6-f95222a63f9e None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:10.787063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d65480-37c7-43cf-bae6-f95222a63f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:10.787341 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d65480-37c7-43cf-bae6-f95222a63f9e None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:10.788091 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d65480-37c7-43cf-bae6-f95222a63f9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:10.788404 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d65480-37c7-43cf-bae6-f95222a63f9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:10.789770 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d65480-37c7-43cf-bae6-f95222a63f9e None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:10.802212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 618/1243] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:10.808028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8b46d-103a-4090-88bc-9bd36657d8eb None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:10.884463 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:10.906774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8b46d-103a-4090-88bc-9bd36657d8eb None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:10.941201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af500ad4-a18b-46b7-a830-6d3ccde72659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:10.941998 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af500ad4-a18b-46b7-a830-6d3ccde72659 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:10.942509 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af500ad4-a18b-46b7-a830-6d3ccde72659 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:10.943201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af500ad4-a18b-46b7-a830-6d3ccde72659 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:10.950615 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af500ad4-a18b-46b7-a830-6d3ccde72659 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:10.952119 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af500ad4-a18b-46b7-a830-6d3ccde72659 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:10.966308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-af500ad4-a18b-46b7-a830-6d3ccde72659 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:17:10.966308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:10.966308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:10.966308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:10.966308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:10.966308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:10.966308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:10.966308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:10.966308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:10.966308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:10.966308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:10.966308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:10.966308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:17:10.966308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:17:10.966308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:17:10.967382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:17:10.967382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:10.967382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:10.967382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:10.967382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:10.967382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:10.967382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:10.967382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:10.967382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:10.967382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:10.967382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:10.967382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:10.967382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:10.967382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:10.967382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:10.967382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:10.967382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:10.968474 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:17:10.968474 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:17:10.968474 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:17:10.968474 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:17:10.968474 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:17:10.968474 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:17:10.968474 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:17:10.968474 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:17:10.968474 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 626/1244] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:17:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:10.973423 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d65480-37c7-43cf-bae6-f95222a63f9e None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:11.080357 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d65480-37c7-43cf-bae6-f95222a63f9e None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:11.116115 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab9840b-2574-4502-8c8e-865c0f060b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:11.116775 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9840b-2574-4502-8c8e-865c0f060b60 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:11.117062 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9840b-2574-4502-8c8e-865c0f060b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:11.117343 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab9840b-2574-4502-8c8e-865c0f060b60 None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:11.118121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab9840b-2574-4502-8c8e-865c0f060b60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:11.118432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab9840b-2574-4502-8c8e-865c0f060b60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:11.119955 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab9840b-2574-4502-8c8e-865c0f060b60 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:11.132221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 619/1245] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:17:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:11.139350 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af500ad4-a18b-46b7-a830-6d3ccde72659 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:11.243914 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af500ad4-a18b-46b7-a830-6d3ccde72659 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:11.279763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41e9df-c49e-43d7-9615-3069cd5212f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:11.280405 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41e9df-c49e-43d7-9615-3069cd5212f2 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:11.280692 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41e9df-c49e-43d7-9615-3069cd5212f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:11.280971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41e9df-c49e-43d7-9615-3069cd5212f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:11.281791 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41e9df-c49e-43d7-9615-3069cd5212f2 None admin] RBAC: Adding query filter params (role.id=57bc5df632b3407d97116ae34d92a5cb, scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d, user.id=a932a9390fdd47f3adcf053a60250944) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:11.282105 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41e9df-c49e-43d7-9615-3069cd5212f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:11.283724 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41e9df-c49e-43d7-9615-3069cd5212f2 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:11.289633 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 627/1246] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:17:11 2022] GET /identity/v3/role_assignments?user.id=a932a9390fdd47f3adcf053a60250944&scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d&role.id=57bc5df632b3407d97116ae34d92a5cb => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:13.420980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52448b2c-bdd9-4f9b-bd38-1911d9742713 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:13.421564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52448b2c-bdd9-4f9b-bd38-1911d9742713 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:13.421564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52448b2c-bdd9-4f9b-bd38-1911d9742713 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:13.422888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 620/1247] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:17:13.428206 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a06733-f24d-40ff-ad56-399dfaf8db5b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:13.428583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a06733-f24d-40ff-ad56-399dfaf8db5b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:13.428766 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a06733-f24d-40ff-ad56-399dfaf8db5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:13.433832 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:13.445603 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-73a06733-f24d-40ff-ad56-399dfaf8db5b None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:17:13.446780 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-73a06733-f24d-40ff-ad56-399dfaf8db5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:17:13.515356 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 628/1248] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:17:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 16:17:13.539418 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52448b2c-bdd9-4f9b-bd38-1911d9742713 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:13.557269 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:13.640124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52448b2c-bdd9-4f9b-bd38-1911d9742713 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:13.673200 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a267d-430d-4f03-a514-49f03e60eed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:13.673905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a267d-430d-4f03-a514-49f03e60eed3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:13.674189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a267d-430d-4f03-a514-49f03e60eed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:13.674466 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a267d-430d-4f03-a514-49f03e60eed3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:13.684967 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a267d-430d-4f03-a514-49f03e60eed3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:17:13.692660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-a14a267d-430d-4f03-a514-49f03e60eed3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:17:13.692660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:13.692660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:13.692660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:13.692660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:13.692660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:13.692660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:13.692660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:13.692660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:13.692660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:13.692660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:13.692660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:13.692660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:17:13.692660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:17:13.692660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:17:13.692660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:17:13.696224 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:17:13.696224 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:17:13.696224 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:17:13.696224 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:17:13.696224 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:13.696224 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:13.696224 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:13.696224 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:13.696224 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:13.696224 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:13.696224 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:13.696224 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:13.696224 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:13.696224 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:13.696224 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:13.696224 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:13.697298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:13.697298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:13.697298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:13.697298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:13.697298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:17:13.697298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:17:13.697298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:17:13.697298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:17:13.697298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:17:13.697298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:17:13.697298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:17:13.697298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:17:13.697298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 621/1249] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:17:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:13.700868 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a06733-f24d-40ff-ad56-399dfaf8db5b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:13.803747 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a06733-f24d-40ff-ad56-399dfaf8db5b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:13.833520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:13.838305 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802b333-dcd5-4ef4-9cee-ffa6e3e31d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:13.838984 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802b333-dcd5-4ef4-9cee-ffa6e3e31d18 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:13.839270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802b333-dcd5-4ef4-9cee-ffa6e3e31d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:13.839606 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802b333-dcd5-4ef4-9cee-ffa6e3e31d18 None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:13.840346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802b333-dcd5-4ef4-9cee-ffa6e3e31d18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:13.840681 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802b333-dcd5-4ef4-9cee-ffa6e3e31d18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:13.842069 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802b333-dcd5-4ef4-9cee-ffa6e3e31d18 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:13.850203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 629/1250] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:17:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:13.856184 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a267d-430d-4f03-a514-49f03e60eed3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:13.936856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:13.959041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a267d-430d-4f03-a514-49f03e60eed3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:13.992167 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6e1c0-cb27-4fdf-9330-7ba4c7e06ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:13.992988 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6e1c0-cb27-4fdf-9330-7ba4c7e06ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:13.993267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6e1c0-cb27-4fdf-9330-7ba4c7e06ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:13.993545 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6e1c0-cb27-4fdf-9330-7ba4c7e06ab1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:14.006940 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6e1c0-cb27-4fdf-9330-7ba4c7e06ab1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:14.009013 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6e1c0-cb27-4fdf-9330-7ba4c7e06ab1 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:14.027121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-d6c6e1c0-cb27-4fdf-9330-7ba4c7e06ab1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:17:14.027121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:14.027121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:14.027121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:14.027121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:14.027121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:14.027121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:14.027121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:14.027121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:14.027121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:14.027121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:14.027121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:14.027121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:17:14.027121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:17:14.027121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:17:14.028272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:17:14.028272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:14.028272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:14.028272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:14.028272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:14.028272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:14.028272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:14.028272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:14.028272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:14.028272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:14.028272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:14.028272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:14.028272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:14.028272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:14.028272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:14.028272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:14.028272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:14.029494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:17:14.029494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:17:14.029494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:17:14.029494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:17:14.029494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 622/1251] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:17:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:14.034127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802b333-dcd5-4ef4-9cee-ffa6e3e31d18 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:14.136154 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802b333-dcd5-4ef4-9cee-ffa6e3e31d18 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:14.171088 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c360b59-7bf8-4501-8641-92491fca9026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:14.171838 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c360b59-7bf8-4501-8641-92491fca9026 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:14.172126 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c360b59-7bf8-4501-8641-92491fca9026 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:14.172405 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c360b59-7bf8-4501-8641-92491fca9026 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:14.173125 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c360b59-7bf8-4501-8641-92491fca9026 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:14.173452 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c360b59-7bf8-4501-8641-92491fca9026 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:14.174868 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c360b59-7bf8-4501-8641-92491fca9026 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:14.199139 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 630/1252] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:14.199326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6e1c0-cb27-4fdf-9330-7ba4c7e06ab1 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:14.303499 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6e1c0-cb27-4fdf-9330-7ba4c7e06ab1 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:14.329121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:14.336698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626185e7-4776-4ee3-97e9-1533a572eff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:14.337521 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626185e7-4776-4ee3-97e9-1533a572eff3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:14.337781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626185e7-4776-4ee3-97e9-1533a572eff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:14.338057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626185e7-4776-4ee3-97e9-1533a572eff3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:14.345960 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626185e7-4776-4ee3-97e9-1533a572eff3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:14.347387 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626185e7-4776-4ee3-97e9-1533a572eff3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:14.361971 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-626185e7-4776-4ee3-97e9-1533a572eff3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 16:17:14.361971 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:14.361971 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:14.361971 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:14.361971 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:14.361971 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:14.361971 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:14.361971 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:14.361971 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:14.361971 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:14.361971 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:14.361971 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:14.361971 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:17:14.361971 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:17:14.361971 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:17:14.361971 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:17:14.363110 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:14.363110 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:14.363110 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:17:14.363110 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:17:14.363110 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:17:14.363110 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:17:14.363110 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:14.363110 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:14.363110 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:14.363110 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:14.363110 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:14.363110 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:14.363110 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:14.363110 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:14.363110 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:14.363110 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:14.363110 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:14.363110 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:14.364544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:14.364544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:14.364544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:17:14.364544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:17:14.364544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:17:14.364544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:17:14.364544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:17:14.364544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:17:14.364544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 16:17:14.364544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:17:14.364544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 623/1253] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:17:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:14.369208 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c360b59-7bf8-4501-8641-92491fca9026 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:14.425475 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:14.470461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c360b59-7bf8-4501-8641-92491fca9026 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:14.503427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b83ce2-093f-4abe-a352-c4f7b406f529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:14.508034 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b83ce2-093f-4abe-a352-c4f7b406f529 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:14.508034 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b83ce2-093f-4abe-a352-c4f7b406f529 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:14.508034 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b83ce2-093f-4abe-a352-c4f7b406f529 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:14.508034 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b83ce2-093f-4abe-a352-c4f7b406f529 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:14.508034 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b83ce2-093f-4abe-a352-c4f7b406f529 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:14.508034 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b83ce2-093f-4abe-a352-c4f7b406f529 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:14.524264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 631/1254] 10.222.0.67 () {56 vars in 1143 bytes} [Thu Jun 30 16:17:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:14.531102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626185e7-4776-4ee3-97e9-1533a572eff3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:14.634207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626185e7-4776-4ee3-97e9-1533a572eff3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:14.669713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ef754-9250-44fb-8fa0-d415d7bd24c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:14.670352 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ef754-9250-44fb-8fa0-d415d7bd24c3 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:14.670636 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ef754-9250-44fb-8fa0-d415d7bd24c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:14.670911 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4ef754-9250-44fb-8fa0-d415d7bd24c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:14.726424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:14.738714 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ef754-9250-44fb-8fa0-d415d7bd24c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:14.740260 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4ef754-9250-44fb-8fa0-d415d7bd24c3 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:14.837502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 624/1255] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:17:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 16:17:14.844169 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b83ce2-093f-4abe-a352-c4f7b406f529 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:14.943906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b83ce2-093f-4abe-a352-c4f7b406f529 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:14.957758 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:14.977696 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51d5e7-ea1d-4075-a210-4b247bebae31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:14.978420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51d5e7-ea1d-4075-a210-4b247bebae31 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:14.978703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51d5e7-ea1d-4075-a210-4b247bebae31 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:14.978976 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c51d5e7-ea1d-4075-a210-4b247bebae31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:14.989320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51d5e7-ea1d-4075-a210-4b247bebae31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:14.990826 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c51d5e7-ea1d-4075-a210-4b247bebae31 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:15.010079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-0c51d5e7-ea1d-4075-a210-4b247bebae31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:17:15.010079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:15.010079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:15.010079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:15.010079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:15.010079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:15.010079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:15.010079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:15.010079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:15.010079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:15.010079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:15.010079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:15.010079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:17:15.010079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:17:15.010079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:17:15.011064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:17:15.011064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:15.011064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:15.011064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:15.011064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:15.011064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:15.011064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:15.011064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:15.011064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:15.011064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:15.011064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:15.011064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:15.011064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:15.011064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:15.011064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:15.011064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:15.011064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:15.011989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:17:15.011989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:17:15.011989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:17:15.011989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:17:15.011989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 632/1256] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:17:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:15.018124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ef754-9250-44fb-8fa0-d415d7bd24c3 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:15.105435 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:15.117114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4ef754-9250-44fb-8fa0-d415d7bd24c3 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:15.151762 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26182c6c-46ba-4153-a760-e80de955019d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:15.152420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26182c6c-46ba-4153-a760-e80de955019d None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:15.152707 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26182c6c-46ba-4153-a760-e80de955019d None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:15.152985 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26182c6c-46ba-4153-a760-e80de955019d None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:15.153832 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26182c6c-46ba-4153-a760-e80de955019d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:15.154155 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26182c6c-46ba-4153-a760-e80de955019d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:15.155602 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26182c6c-46ba-4153-a760-e80de955019d None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:15.166628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 625/1257] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:15.172613 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51d5e7-ea1d-4075-a210-4b247bebae31 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:15.275816 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c51d5e7-ea1d-4075-a210-4b247bebae31 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:15.309406 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94b885-bc3d-4a5d-87cf-a4c8cbcb1744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:15.310853 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94b885-bc3d-4a5d-87cf-a4c8cbcb1744 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:15.311136 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94b885-bc3d-4a5d-87cf-a4c8cbcb1744 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:15.311451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94b885-bc3d-4a5d-87cf-a4c8cbcb1744 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:15.318346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94b885-bc3d-4a5d-87cf-a4c8cbcb1744 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:15.319835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94b885-bc3d-4a5d-87cf-a4c8cbcb1744 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:15.334502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-3f94b885-bc3d-4a5d-87cf-a4c8cbcb1744 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:17:15.334502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:15.334502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:15.334502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:15.334502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:15.334502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:15.334502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:15.334502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:15.334502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:15.334502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:15.334502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:15.334502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:15.334502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:17:15.334502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:17:15.334502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:17:15.341292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:17:15.341292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:15.341292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:15.341292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:15.341292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:15.341292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:15.341292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:15.341292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:15.341292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:15.341292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:15.341292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:15.341292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:15.341292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:15.341292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:15.341292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:15.341292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:15.341292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:15.342168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:17:15.342168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:17:15.342168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:17:15.342168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:17:15.342168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:17:15.342168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:17:15.342168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:17:15.342168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:17:15.342168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 633/1258] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:17:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:15.342651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26182c6c-46ba-4153-a760-e80de955019d None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:15.444185 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26182c6c-46ba-4153-a760-e80de955019d None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:15.477552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf92e91-670f-4708-aab1-e1eccb9718cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:15.478284 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf92e91-670f-4708-aab1-e1eccb9718cd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:15.478568 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf92e91-670f-4708-aab1-e1eccb9718cd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:15.478845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf92e91-670f-4708-aab1-e1eccb9718cd None admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:15.479659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf92e91-670f-4708-aab1-e1eccb9718cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:15.479972 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf92e91-670f-4708-aab1-e1eccb9718cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:15.481989 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf92e91-670f-4708-aab1-e1eccb9718cd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:15.493802 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 626/1259] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:17:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:15.500934 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94b885-bc3d-4a5d-87cf-a4c8cbcb1744 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:15.526743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:15.600578 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94b885-bc3d-4a5d-87cf-a4c8cbcb1744 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:15.635458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae93931-0638-4c35-aa4b-9cd7d730c553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:15.636130 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae93931-0638-4c35-aa4b-9cd7d730c553 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:15.636418 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae93931-0638-4c35-aa4b-9cd7d730c553 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:15.636702 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae93931-0638-4c35-aa4b-9cd7d730c553 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:15.637632 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae93931-0638-4c35-aa4b-9cd7d730c553 None admin] RBAC: Adding query filter params (role.id=8acc34a163874a60b03728b814eaf075, scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d, user.id=a932a9390fdd47f3adcf053a60250944) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:15.638117 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae93931-0638-4c35-aa4b-9cd7d730c553 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:15.639585 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae93931-0638-4c35-aa4b-9cd7d730c553 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:15.645500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 634/1260] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:17:15 2022] GET /identity/v3/role_assignments?user.id=a932a9390fdd47f3adcf053a60250944&scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d&role.id=8acc34a163874a60b03728b814eaf075 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:17.681791 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c37a6-2684-491d-986d-95bfcefad3ae None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:17.682381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c37a6-2684-491d-986d-95bfcefad3ae None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:17.682381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c37a6-2684-491d-986d-95bfcefad3ae None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:17.683783 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 627/1261] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:17:17.689495 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f0b5f-fcb2-4e45-808e-91e865c220e6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:17.689899 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f0b5f-fcb2-4e45-808e-91e865c220e6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:17.690058 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f0b5f-fcb2-4e45-808e-91e865c220e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:17.706867 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-768f0b5f-fcb2-4e45-808e-91e865c220e6 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:17:17.708185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-768f0b5f-fcb2-4e45-808e-91e865c220e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:17:17.774484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 635/1262] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:17:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:17:17.798076 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c37a6-2684-491d-986d-95bfcefad3ae None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:17.808834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:17.901546 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c37a6-2684-491d-986d-95bfcefad3ae None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:17.934377 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa022e1b-9869-4c4b-852f-46a45b6c0654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:17.935697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa022e1b-9869-4c4b-852f-46a45b6c0654 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:17.935978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa022e1b-9869-4c4b-852f-46a45b6c0654 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:17.936252 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa022e1b-9869-4c4b-852f-46a45b6c0654 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:17.947303 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa022e1b-9869-4c4b-852f-46a45b6c0654 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:17:17.955114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-aa022e1b-9869-4c4b-852f-46a45b6c0654 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:17:17.955114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:17.955114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:17.955114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:17.955114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:17.955114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:17.955114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:17.955114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:17.955114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:17.955114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:17.955114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:17.955114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:17.955114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:17:17.955114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:17:17.955114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:17:17.955114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:17:17.956360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:17:17.956360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:17:17.956360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:17:17.956360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:17:17.956360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:17.956360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:17.956360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:17.956360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:17.956360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:17.956360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:17.956360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:17.956360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:17.956360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:17.956360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:17.956360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:17.956360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:17.957651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:17.957651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:17.957651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:17.957651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:17.957651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:17:17.957651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:17:17.957651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:17:17.957651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:17:17.957651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:17:17.957651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:17:17.957651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:17:17.957651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:17:17.957651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 628/1263] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:17:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:17.962647 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f0b5f-fcb2-4e45-808e-91e865c220e6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:17.981681 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:18.061360 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f0b5f-fcb2-4e45-808e-91e865c220e6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:18.095489 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb0c28-5976-4043-94ed-ab132d1cef6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:18.096181 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb0c28-5976-4043-94ed-ab132d1cef6e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:18.096658 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb0c28-5976-4043-94ed-ab132d1cef6e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:18.097588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb0c28-5976-4043-94ed-ab132d1cef6e None admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:18.098330 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb0c28-5976-4043-94ed-ab132d1cef6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:18.098642 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb0c28-5976-4043-94ed-ab132d1cef6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:18.100067 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb0c28-5976-4043-94ed-ab132d1cef6e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:18.108995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 636/1264] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:17:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:17:18.114949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa022e1b-9869-4c4b-852f-46a45b6c0654 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:18.182982 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:18.216595 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa022e1b-9869-4c4b-852f-46a45b6c0654 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:18.250342 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86686b-6c7f-4616-b8af-e6320a983ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:18.251167 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86686b-6c7f-4616-b8af-e6320a983ecd None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:18.251451 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86686b-6c7f-4616-b8af-e6320a983ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:18.251760 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86686b-6c7f-4616-b8af-e6320a983ecd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:18.261970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86686b-6c7f-4616-b8af-e6320a983ecd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:18.263779 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86686b-6c7f-4616-b8af-e6320a983ecd None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:18.281892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-fa86686b-6c7f-4616-b8af-e6320a983ecd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:17:18.281892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:18.281892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:18.281892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:18.281892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:18.281892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:18.281892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:18.281892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:18.281892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:18.281892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:18.281892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:18.281892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:18.281892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:17:18.281892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:17:18.281892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:17:18.282991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:17:18.282991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:18.282991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:18.282991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:18.282991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:18.282991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:18.282991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:18.282991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:18.282991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:18.282991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:18.282991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:18.282991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:18.282991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:18.282991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:18.282991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:18.282991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:18.282991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:18.284153 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:17:18.284153 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:17:18.284153 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:17:18.284153 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:17:18.284153 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 629/1265] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:18.289460 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb0c28-5976-4043-94ed-ab132d1cef6e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:18.371412 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:18.391608 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb0c28-5976-4043-94ed-ab132d1cef6e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:18.424928 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2696eabe-b91c-46b5-88a4-4c0365d4eb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:18.425587 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696eabe-b91c-46b5-88a4-4c0365d4eb84 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:18.425872 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696eabe-b91c-46b5-88a4-4c0365d4eb84 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:18.426153 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696eabe-b91c-46b5-88a4-4c0365d4eb84 None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:18.426874 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2696eabe-b91c-46b5-88a4-4c0365d4eb84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:18.427186 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2696eabe-b91c-46b5-88a4-4c0365d4eb84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:18.428607 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2696eabe-b91c-46b5-88a4-4c0365d4eb84 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:18.440101 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 637/1266] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:18.446097 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86686b-6c7f-4616-b8af-e6320a983ecd None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:18.549111 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86686b-6c7f-4616-b8af-e6320a983ecd None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:18.569531 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:18.584107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16b456-d7bc-4d06-8349-95a15f092aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:18.584905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16b456-d7bc-4d06-8349-95a15f092aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:18.589445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16b456-d7bc-4d06-8349-95a15f092aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:18.589445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16b456-d7bc-4d06-8349-95a15f092aa5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:18.593516 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16b456-d7bc-4d06-8349-95a15f092aa5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:18.594958 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16b456-d7bc-4d06-8349-95a15f092aa5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:18.609151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-3f16b456-d7bc-4d06-8349-95a15f092aa5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 16:17:18.609151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:18.609151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:18.609151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:18.609151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:18.609151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:18.609151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:18.609151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:18.609151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:18.609151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:18.609151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:18.609151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:18.609151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:17:18.609151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:17:18.609151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:17:18.609151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:17:18.610420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:18.610420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:18.610420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:17:18.610420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:17:18.610420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:17:18.610420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:17:18.610420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:18.610420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:18.610420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:18.610420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:18.610420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:18.610420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:18.610420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:18.610420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:18.610420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:18.610420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:18.610420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:18.610420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:18.611636 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:18.611636 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:18.611636 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:17:18.611636 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:17:18.611636 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:17:18.611636 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:17:18.611636 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:17:18.611636 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:17:18.611636 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 16:17:18.611636 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:17:18.611636 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 630/1267] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:17:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:18.616320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2696eabe-b91c-46b5-88a4-4c0365d4eb84 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:18.715997 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2696eabe-b91c-46b5-88a4-4c0365d4eb84 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:18.749335 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10a27f-4d3f-49ca-8514-9a9d07bf4845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:18.749998 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10a27f-4d3f-49ca-8514-9a9d07bf4845 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:18.750277 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10a27f-4d3f-49ca-8514-9a9d07bf4845 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:18.750553 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d10a27f-4d3f-49ca-8514-9a9d07bf4845 None admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:18.751377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d10a27f-4d3f-49ca-8514-9a9d07bf4845 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:18.751711 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d10a27f-4d3f-49ca-8514-9a9d07bf4845 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:18.753225 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d10a27f-4d3f-49ca-8514-9a9d07bf4845 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:18.762098 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:18.770720 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 638/1268] 10.222.0.67 () {56 vars in 1143 bytes} [Thu Jun 30 16:17:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:17:18.776492 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16b456-d7bc-4d06-8349-95a15f092aa5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:18.876610 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16b456-d7bc-4d06-8349-95a15f092aa5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:18.911000 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33e822-593f-4bfe-b654-bead6c35b1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:18.911892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33e822-593f-4bfe-b654-bead6c35b1b6 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:18.912163 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33e822-593f-4bfe-b654-bead6c35b1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:18.912436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33e822-593f-4bfe-b654-bead6c35b1b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:18.923062 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33e822-593f-4bfe-b654-bead6c35b1b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:18.924536 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af33e822-593f-4bfe-b654-bead6c35b1b6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:18.942454 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-af33e822-593f-4bfe-b654-bead6c35b1b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:17:18.942454 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:18.942454 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:18.942454 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:18.942454 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:18.942454 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:18.942454 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:18.942454 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:18.942454 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:18.942454 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:18.942454 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:18.942454 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:18.942454 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:17:18.942454 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:17:18.942454 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:17:18.946165 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:17:18.946165 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:18.946165 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:18.946165 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:18.946165 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:18.946165 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:18.946165 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:18.946165 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:18.946165 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:18.946165 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:18.946165 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:18.946165 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:18.946165 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:18.946165 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:18.946165 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:18.946165 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:18.946165 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:18.947286 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:17:18.947286 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:17:18.947286 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:17:18.947286 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:17:18.947286 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 631/1269] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:17:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:18.952148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10a27f-4d3f-49ca-8514-9a9d07bf4845 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:19.051654 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d10a27f-4d3f-49ca-8514-9a9d07bf4845 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:19.085820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9374f5-e01b-4c14-b50b-76202933be0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:19.086469 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9374f5-e01b-4c14-b50b-76202933be0e None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:19.086753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9374f5-e01b-4c14-b50b-76202933be0e None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:19.087029 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9374f5-e01b-4c14-b50b-76202933be0e None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:19.087758 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9374f5-e01b-4c14-b50b-76202933be0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:19.088064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9374f5-e01b-4c14-b50b-76202933be0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:19.089437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9374f5-e01b-4c14-b50b-76202933be0e None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:19.101201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 639/1270] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:19.107512 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33e822-593f-4bfe-b654-bead6c35b1b6 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:19.143296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:19.207844 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af33e822-593f-4bfe-b654-bead6c35b1b6 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:19.243352 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de7184-edb8-4371-af0d-5cf3d7b6e899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:19.244172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de7184-edb8-4371-af0d-5cf3d7b6e899 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:19.244458 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de7184-edb8-4371-af0d-5cf3d7b6e899 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:19.244739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de7184-edb8-4371-af0d-5cf3d7b6e899 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:19.252487 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7de7184-edb8-4371-af0d-5cf3d7b6e899 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:19.254027 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7de7184-edb8-4371-af0d-5cf3d7b6e899 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:19.268445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f7de7184-edb8-4371-af0d-5cf3d7b6e899 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:17:19.268445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:19.268445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:19.268445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:19.268445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:19.268445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:19.268445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:19.268445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:19.268445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:19.268445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:19.268445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:19.268445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:19.268445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:17:19.268445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:17:19.268445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:17:19.269720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:17:19.269720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:19.269720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:19.269720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:19.269720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:19.269720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:19.269720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:19.269720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:19.269720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:19.269720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:19.269720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:19.269720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:19.269720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:19.269720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:19.269720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:19.269720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:19.269720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:19.270815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:17:19.270815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:17:19.270815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:17:19.270815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:17:19.270815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:17:19.270815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:17:19.270815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:17:19.270815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:17:19.270815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 632/1271] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:17:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:19.275632 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9374f5-e01b-4c14-b50b-76202933be0e None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:19.335121 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:19.383076 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9374f5-e01b-4c14-b50b-76202933be0e None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:19.417825 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00e4eb-8ccf-4cfc-b7fe-accc410dbe4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:19.418408 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00e4eb-8ccf-4cfc-b7fe-accc410dbe4f None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:19.418695 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00e4eb-8ccf-4cfc-b7fe-accc410dbe4f None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:19.418977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00e4eb-8ccf-4cfc-b7fe-accc410dbe4f None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:19.420420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00e4eb-8ccf-4cfc-b7fe-accc410dbe4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:19.420732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00e4eb-8ccf-4cfc-b7fe-accc410dbe4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:19.422176 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00e4eb-8ccf-4cfc-b7fe-accc410dbe4f None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:19.434252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 640/1272] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:17:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:19.441424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de7184-edb8-4371-af0d-5cf3d7b6e899 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:19.534833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:19.543287 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de7184-edb8-4371-af0d-5cf3d7b6e899 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:19.577000 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c33fbd-232e-4aba-b868-fd4a929a5741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:19.577982 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c33fbd-232e-4aba-b868-fd4a929a5741 None admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:19.578266 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c33fbd-232e-4aba-b868-fd4a929a5741 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:19.578544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c33fbd-232e-4aba-b868-fd4a929a5741 None admin] PATH_INFO: `/v3/projects/19ef92ebcce94b90bf39ad3a58772a1d/users/a932a9390fdd47f3adcf053a60250944/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:19.580516 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c33fbd-232e-4aba-b868-fd4a929a5741 None admin] RBAC: Authorizing `identity:create_grant(project_id=19ef92ebcce94b90bf39ad3a58772a1d, user_id=a932a9390fdd47f3adcf053a60250944, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:19.581944 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c33fbd-232e-4aba-b868-fd4a929a5741 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:19.756424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 633/1273] 10.222.0.67 () {58 vars in 1439 bytes} [Thu Jun 30 16:17:19 2022] PUT /identity/v3/projects/19ef92ebcce94b90bf39ad3a58772a1d/users/a932a9390fdd47f3adcf053a60250944/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:17:21.840903 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb27efa-b3f5-422d-9afd-3fc4c44c88d8 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:21.841470 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb27efa-b3f5-422d-9afd-3fc4c44c88d8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:21.841632 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb27efa-b3f5-422d-9afd-3fc4c44c88d8 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:21.843143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 641/1274] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:17:21.849627 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a418d2-efef-4bb9-b8f9-c49c3e62ff93 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:21.850132 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a418d2-efef-4bb9-b8f9-c49c3e62ff93 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:21.850132 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a418d2-efef-4bb9-b8f9-c49c3e62ff93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:21.868398 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-01a418d2-efef-4bb9-b8f9-c49c3e62ff93 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:17:21.869663 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-01a418d2-efef-4bb9-b8f9-c49c3e62ff93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:17:21.945634 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 634/1275] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:17:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:17:21.971639 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb27efa-b3f5-422d-9afd-3fc4c44c88d8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:22.086741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb27efa-b3f5-422d-9afd-3fc4c44c88d8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:22.109849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:22.127871 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2df3f3-5c6c-4c65-97bb-996a5afa25e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:22.128752 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2df3f3-5c6c-4c65-97bb-996a5afa25e7 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:22.129127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2df3f3-5c6c-4c65-97bb-996a5afa25e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:22.129492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2df3f3-5c6c-4c65-97bb-996a5afa25e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:22.140975 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2df3f3-5c6c-4c65-97bb-996a5afa25e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 16:17:22.150346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-6e2df3f3-5c6c-4c65-97bb-996a5afa25e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:17:22.150346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:22.150346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:22.150346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:22.150346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:22.150346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:22.150346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:22.150346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:22.150346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:22.150346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:22.150346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:22.150346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:22.150346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 16:17:22.150346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 16:17:22.150346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 16:17:22.150346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 16:17:22.151312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 16:17:22.151312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 16:17:22.151312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 16:17:22.151312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 16:17:22.151312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:22.151312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:22.151312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:22.151312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:22.151312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:22.151312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:22.151312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:22.151312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:22.151312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:22.151312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:22.151312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:22.151312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:22.152197 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:22.152197 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:22.152197 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:22.152197 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:22.152197 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 16:17:22.152197 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 16:17:22.152197 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 16:17:22.152197 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 16:17:22.152197 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 16:17:22.152197 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 16:17:22.152197 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 16:17:22.152197 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:17:22.152907 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 642/1276] 10.222.0.67 () {56 vars in 1096 bytes} [Thu Jun 30 16:17:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:22.161422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a418d2-efef-4bb9-b8f9-c49c3e62ff93 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:22.258148 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:22.278209 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a418d2-efef-4bb9-b8f9-c49c3e62ff93 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:22.314552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069e86d-6b32-49bd-8f0c-5db4f1534c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:22.315236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069e86d-6b32-49bd-8f0c-5db4f1534c03 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:22.315767 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069e86d-6b32-49bd-8f0c-5db4f1534c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:22.316289 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069e86d-6b32-49bd-8f0c-5db4f1534c03 None admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:22.317251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0069e86d-6b32-49bd-8f0c-5db4f1534c03 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:22.317562 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0069e86d-6b32-49bd-8f0c-5db4f1534c03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:22.319560 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0069e86d-6b32-49bd-8f0c-5db4f1534c03 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:22.331257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 635/1277] 10.222.0.67 () {56 vars in 1099 bytes} [Thu Jun 30 16:17:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:17:22.337306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2df3f3-5c6c-4c65-97bb-996a5afa25e7 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:22.460613 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2df3f3-5c6c-4c65-97bb-996a5afa25e7 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:22.498885 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc8e06-6339-4a17-b96a-ecefed735201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:22.499717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc8e06-6339-4a17-b96a-ecefed735201 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:22.500004 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc8e06-6339-4a17-b96a-ecefed735201 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:22.500283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc8e06-6339-4a17-b96a-ecefed735201 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:22.512786 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc8e06-6339-4a17-b96a-ecefed735201 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:22.514471 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc8e06-6339-4a17-b96a-ecefed735201 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:22.539622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-3abc8e06-6339-4a17-b96a-ecefed735201 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:17:22.539622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:22.539622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:22.539622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:22.539622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:22.539622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:22.539622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:22.539622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:22.539622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:22.539622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:22.539622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:22.539622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:22.539622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:17:22.539622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:17:22.539622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:17:22.540746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:17:22.540746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:22.540746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:22.540746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:22.540746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:22.540746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:22.540746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:22.540746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:22.540746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:22.540746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:22.540746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:22.540746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:22.540746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:22.540746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:22.540746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:22.540746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:22.540746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:22.541851 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:17:22.541851 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:17:22.541851 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:17:22.541851 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:17:22.541851 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 643/1278] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:17:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:22.547890 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069e86d-6b32-49bd-8f0c-5db4f1534c03 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:22.652866 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069e86d-6b32-49bd-8f0c-5db4f1534c03 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:22.692457 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4032273-52bd-49bb-9a53-199af290fe0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:22.693127 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4032273-52bd-49bb-9a53-199af290fe0c None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:22.693411 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4032273-52bd-49bb-9a53-199af290fe0c None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:22.693684 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4032273-52bd-49bb-9a53-199af290fe0c None admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:22.694408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4032273-52bd-49bb-9a53-199af290fe0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:22.694709 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4032273-52bd-49bb-9a53-199af290fe0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:22.696119 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4032273-52bd-49bb-9a53-199af290fe0c None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:22.708231 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 636/1279] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:17:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:22.714003 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc8e06-6339-4a17-b96a-ecefed735201 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:22.745617 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:22.824566 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc8e06-6339-4a17-b96a-ecefed735201 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:22.860391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3993ff-69d7-4060-8267-32885f9bf355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:22.861189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3993ff-69d7-4060-8267-32885f9bf355 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:22.861475 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3993ff-69d7-4060-8267-32885f9bf355 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:22.861754 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3993ff-69d7-4060-8267-32885f9bf355 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:22.869140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3993ff-69d7-4060-8267-32885f9bf355 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:22.870597 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3993ff-69d7-4060-8267-32885f9bf355 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:22.885796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-0d3993ff-69d7-4060-8267-32885f9bf355 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 16:17:22.885796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:22.885796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:22.885796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:22.885796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:22.885796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:22.885796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:22.885796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:22.885796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:22.885796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:22.885796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:22.885796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:22.885796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 16:17:22.885796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 16:17:22.885796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 16:17:22.885796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 16:17:22.887734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:22.887734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:22.887734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 16:17:22.887734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:17:22.887734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 16:17:22.887734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 16:17:22.887734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:22.887734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:22.887734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:22.887734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:22.887734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:22.887734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:22.887734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:22.887734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:22.887734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:22.887734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:22.887734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:22.887734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:22.890710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:22.890710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:22.890710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 16:17:22.890710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 16:17:22.890710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 16:17:22.890710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 16:17:22.890710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 16:17:22.890710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 16:17:22.890710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 16:17:22.890710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:17:22.890710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 644/1280] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:17:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:22.895713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4032273-52bd-49bb-9a53-199af290fe0c None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:22.899681 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:22.995326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4032273-52bd-49bb-9a53-199af290fe0c None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:23.032574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a12903-2f8c-4e2c-bae5-dc4d5ad2a1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:23.033245 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a12903-2f8c-4e2c-bae5-dc4d5ad2a1bf None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:23.033532 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a12903-2f8c-4e2c-bae5-dc4d5ad2a1bf None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:23.033809 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a12903-2f8c-4e2c-bae5-dc4d5ad2a1bf None admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:23.034643 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a12903-2f8c-4e2c-bae5-dc4d5ad2a1bf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:23.035053 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a12903-2f8c-4e2c-bae5-dc4d5ad2a1bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:23.036952 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a12903-2f8c-4e2c-bae5-dc4d5ad2a1bf None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:23.055579 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 637/1281] 10.222.0.67 () {56 vars in 1143 bytes} [Thu Jun 30 16:17:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:23.061838 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3993ff-69d7-4060-8267-32885f9bf355 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:23.156564 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:23.166011 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3993ff-69d7-4060-8267-32885f9bf355 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:23.200403 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ffae0-aa7c-48d3-a43a-bd32f38c674c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:23.202259 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ffae0-aa7c-48d3-a43a-bd32f38c674c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:23.202542 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ffae0-aa7c-48d3-a43a-bd32f38c674c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:23.202824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ffae0-aa7c-48d3-a43a-bd32f38c674c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:23.271079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ffae0-aa7c-48d3-a43a-bd32f38c674c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:23.272544 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ffae0-aa7c-48d3-a43a-bd32f38c674c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:23.363325 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:23.373564 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 645/1282] 10.222.0.67 () {58 vars in 1282 bytes} [Thu Jun 30 16:17:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 16:17:23.379630 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a12903-2f8c-4e2c-bae5-dc4d5ad2a1bf None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:23.445287 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:23.483177 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a12903-2f8c-4e2c-bae5-dc4d5ad2a1bf None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:23.520315 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b4243-d2cb-4d8c-8312-3a9dd421ca3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:23.521156 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b4243-d2cb-4d8c-8312-3a9dd421ca3a None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:23.521384 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b4243-d2cb-4d8c-8312-3a9dd421ca3a None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:23.521652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b4243-d2cb-4d8c-8312-3a9dd421ca3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:23.531978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b4243-d2cb-4d8c-8312-3a9dd421ca3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:23.533416 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b4243-d2cb-4d8c-8312-3a9dd421ca3a None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:23.550606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-f52b4243-d2cb-4d8c-8312-3a9dd421ca3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:17:23.550606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:23.550606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:23.550606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:23.550606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:23.550606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:23.550606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:23.550606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:23.550606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:23.550606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:23.550606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:23.550606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:23.550606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 16:17:23.550606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 16:17:23.550606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 16:17:23.551772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 16:17:23.551772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:23.551772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:23.551772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:23.551772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:23.551772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:23.551772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:23.551772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:23.551772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:23.551772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:23.551772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:23.551772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:23.551772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:23.551772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:23.551772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:23.551772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:23.551772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:23.557777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 16:17:23.557777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 16:17:23.557777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 16:17:23.557777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:17:23.557777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 638/1283] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:17:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:23.558129 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ffae0-aa7c-48d3-a43a-bd32f38c674c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:23.659890 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ffae0-aa7c-48d3-a43a-bd32f38c674c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:23.694059 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ea269-b55c-4626-8d0b-f0ca94ce665d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:23.694624 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ea269-b55c-4626-8d0b-f0ca94ce665d None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:23.694901 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ea269-b55c-4626-8d0b-f0ca94ce665d None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:23.695247 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ea269-b55c-4626-8d0b-f0ca94ce665d None admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:23.695973 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4ea269-b55c-4626-8d0b-f0ca94ce665d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:23.696277 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4ea269-b55c-4626-8d0b-f0ca94ce665d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:23.697652 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4ea269-b55c-4626-8d0b-f0ca94ce665d None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:23.709288 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 646/1284] 10.222.0.67 () {56 vars in 1109 bytes} [Thu Jun 30 16:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:23.714957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b4243-d2cb-4d8c-8312-3a9dd421ca3a None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:23.823323 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b4243-d2cb-4d8c-8312-3a9dd421ca3a None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:23.838615 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:23.858842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244fef7-acc2-41d1-9991-195b4e9dcad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:23.859659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244fef7-acc2-41d1-9991-195b4e9dcad4 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:23.859981 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244fef7-acc2-41d1-9991-195b4e9dcad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:23.860220 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244fef7-acc2-41d1-9991-195b4e9dcad4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:23.866723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244fef7-acc2-41d1-9991-195b4e9dcad4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:23.868206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244fef7-acc2-41d1-9991-195b4e9dcad4 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:23.882703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application [None req-b244fef7-acc2-41d1-9991-195b4e9dcad4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:17:23.882703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:23.882703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:23.882703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:23.882703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:23.882703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:23.882703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:23.882703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:23.882703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:23.882703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:23.882703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:23.882703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:23.882703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 16:17:23.882703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 16:17:23.882703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 16:17:23.883884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 16:17:23.883884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:23.883884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:23.883884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:23.883884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:23.883884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:23.883884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:23.883884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:23.883884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:23.883884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:23.883884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:23.883884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:23.883884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:23.883884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:23.883884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:23.883884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:23.883884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:23.884916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 16:17:23.884916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 16:17:23.884916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 16:17:23.884916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 16:17:23.884916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 16:17:23.884916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 16:17:23.884916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 16:17:23.884916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: ERROR keystone.server.flask.application Jun 30 16:17:23.884916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 639/1285] 10.222.0.67 () {56 vars in 1111 bytes} [Thu Jun 30 16:17:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:23.890120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ea269-b55c-4626-8d0b-f0ca94ce665d None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:23.941222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:23.990649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ea269-b55c-4626-8d0b-f0ca94ce665d None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:24.025529 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018c742-008b-4a7a-a129-1dea2c957877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:24.029533 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018c742-008b-4a7a-a129-1dea2c957877 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:24.029533 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018c742-008b-4a7a-a129-1dea2c957877 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:24.029533 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018c742-008b-4a7a-a129-1dea2c957877 None admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:24.029533 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018c742-008b-4a7a-a129-1dea2c957877 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:24.029533 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018c742-008b-4a7a-a129-1dea2c957877 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:24.030569 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018c742-008b-4a7a-a129-1dea2c957877 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:24.042548 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 647/1286] 10.222.0.67 () {56 vars in 1148 bytes} [Thu Jun 30 16:17:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:24.049670 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244fef7-acc2-41d1-9991-195b4e9dcad4 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:24.153334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244fef7-acc2-41d1-9991-195b4e9dcad4 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:24.188020 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027c1f6-a7fc-48d3-ace7-ffaa2eedb51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:24.188697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027c1f6-a7fc-48d3-ace7-ffaa2eedb51f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:24.188982 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027c1f6-a7fc-48d3-ace7-ffaa2eedb51f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:24.189254 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027c1f6-a7fc-48d3-ace7-ffaa2eedb51f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:24.190074 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027c1f6-a7fc-48d3-ace7-ffaa2eedb51f None admin] RBAC: Adding query filter params (role.id=8acc34a163874a60b03728b814eaf075, scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d, user.id=a932a9390fdd47f3adcf053a60250944) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:24.190385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027c1f6-a7fc-48d3-ace7-ffaa2eedb51f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:24.191859 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027c1f6-a7fc-48d3-ace7-ffaa2eedb51f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:24.197797 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 640/1287] 10.222.0.67 () {56 vars in 1374 bytes} [Thu Jun 30 16:17:24 2022] GET /identity/v3/role_assignments?user.id=a932a9390fdd47f3adcf053a60250944&scope.project.id=19ef92ebcce94b90bf39ad3a58772a1d&role.id=8acc34a163874a60b03728b814eaf075 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:26.305887 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b17df-0502-4674-9559-57cf15322c64 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:26.306489 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b17df-0502-4674-9559-57cf15322c64 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:26.306489 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b17df-0502-4674-9559-57cf15322c64 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:26.307772 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 648/1288] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:17:26.313482 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec58c1-f519-4e33-8e1f-a8fdc834ba45 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:26.313922 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec58c1-f519-4e33-8e1f-a8fdc834ba45 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:26.314023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec58c1-f519-4e33-8e1f-a8fdc834ba45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:26.331312 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-98ec58c1-f519-4e33-8e1f-a8fdc834ba45 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:17:26.332564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-98ec58c1-f519-4e33-8e1f-a8fdc834ba45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:17:26.362002 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:26.399489 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 641/1289] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:17:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:17:26.422172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b17df-0502-4674-9559-57cf15322c64 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:26.526122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b17df-0502-4674-9559-57cf15322c64 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:26.536595 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:26.559237 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962a275-c4d1-4853-92c1-d3ad3698307b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:26.560062 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962a275-c4d1-4853-92c1-d3ad3698307b None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:26.560844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962a275-c4d1-4853-92c1-d3ad3698307b None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:26.561300 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c962a275-c4d1-4853-92c1-d3ad3698307b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:26.562052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962a275-c4d1-4853-92c1-d3ad3698307b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:26.563427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c962a275-c4d1-4853-92c1-d3ad3698307b None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:26.577188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-c962a275-c4d1-4853-92c1-d3ad3698307b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 16:17:26.577188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:26.577188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:17:26.577188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:17:26.577188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:17:26.577188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:17:26.577188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:17:26.577188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:17:26.577188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 16:17:26.577188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:17:26.577188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:17:26.577188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:17:26.577188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:26.577188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:26.577188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:26.577188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:26.577188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:26.578499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:26.578499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:26.578499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:26.578499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:26.578499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:26.578499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:26.578499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:17:26.578499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:17:26.578499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:17:26.578499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:17:26.578499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:26.578499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:26.578499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:26.578499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:26.578499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:26.578499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:26.579414 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:26.579414 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:26.579414 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:26.579414 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:26.579414 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:26.579414 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:26.579414 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:26.579414 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:26.579414 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:26.579414 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:26.579414 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:17:26.579414 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:17:26.579414 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 16:17:26.579414 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:17:26.579414 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 649/1290] 10.222.0.67 () {56 vars in 1117 bytes} [Thu Jun 30 16:17:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:26.585800 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec58c1-f519-4e33-8e1f-a8fdc834ba45 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:26.686582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec58c1-f519-4e33-8e1f-a8fdc834ba45 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:26.719992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fcea8f-bbab-4407-b0e9-4728156da990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:26.720658 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcea8f-bbab-4407-b0e9-4728156da990 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:26.721000 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcea8f-bbab-4407-b0e9-4728156da990 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:26.721282 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fcea8f-bbab-4407-b0e9-4728156da990 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:26.721982 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fcea8f-bbab-4407-b0e9-4728156da990 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:26.722288 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fcea8f-bbab-4407-b0e9-4728156da990 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:26.723714 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fcea8f-bbab-4407-b0e9-4728156da990 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:26.728715 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 642/1291] 10.222.0.67 () {56 vars in 1116 bytes} [Thu Jun 30 16:17:26 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:26.735753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962a275-c4d1-4853-92c1-d3ad3698307b None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:26.836555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c962a275-c4d1-4853-92c1-d3ad3698307b None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:26.869641 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a1296d-277b-4463-a4ca-b3df51382ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:26.870334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1296d-277b-4463-a4ca-b3df51382ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:26.870616 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1296d-277b-4463-a4ca-b3df51382ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:26.870911 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a1296d-277b-4463-a4ca-b3df51382ad0 None admin] PATH_INFO: `/v3/services` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:26.871628 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1296d-277b-4463-a4ca-b3df51382ad0 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:26.871973 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1296d-277b-4463-a4ca-b3df51382ad0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:26.874192 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a1296d-277b-4463-a4ca-b3df51382ad0 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:26.879909 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 650/1292] 10.222.0.67 () {56 vars in 1116 bytes} [Thu Jun 30 16:17:26 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:28.929146 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae7926-5ec9-4a4c-9788-49552b585801 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:28.929709 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae7926-5ec9-4a4c-9788-49552b585801 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:28.929709 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae7926-5ec9-4a4c-9788-49552b585801 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:28.931077 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 643/1293] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:17:28.937013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e5635-ab22-4275-a249-c3cc96cdd680 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:28.937482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e5635-ab22-4275-a249-c3cc96cdd680 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:28.937578 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e5635-ab22-4275-a249-c3cc96cdd680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:28.955019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6a4e5635-ab22-4275-a249-c3cc96cdd680 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:17:28.956232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6a4e5635-ab22-4275-a249-c3cc96cdd680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:17:28.986274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:29.025548 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 651/1294] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:17:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:17:29.049717 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae7926-5ec9-4a4c-9788-49552b585801 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:29.081585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:29.153550 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae7926-5ec9-4a4c-9788-49552b585801 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:29.186974 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0a34e8-e3a8-4e0f-9d49-b5477c89f7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:29.187609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a34e8-e3a8-4e0f-9d49-b5477c89f7a6 None admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:29.187893 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a34e8-e3a8-4e0f-9d49-b5477c89f7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:29.188166 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0a34e8-e3a8-4e0f-9d49-b5477c89f7a6 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:29.188893 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a34e8-e3a8-4e0f-9d49-b5477c89f7a6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:29.190266 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0a34e8-e3a8-4e0f-9d49-b5477c89f7a6 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:29.344296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 644/1295] 10.222.0.67 () {60 vars in 1141 bytes} [Thu Jun 30 16:17:29 2022] POST /identity/v3/services => generated 246 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:17:31.414714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8166a-1f54-4dd2-954d-b67db89d2fc4 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:31.415270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8166a-1f54-4dd2-954d-b67db89d2fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:31.415270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8166a-1f54-4dd2-954d-b67db89d2fc4 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:31.416894 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 652/1296] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:17:31.421963 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41af00-667f-426c-9bca-de7ada943f8c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:31.422359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41af00-667f-426c-9bca-de7ada943f8c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:31.422525 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a41af00-667f-426c-9bca-de7ada943f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:31.440338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9a41af00-667f-426c-9bca-de7ada943f8c None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:17:31.446314 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9a41af00-667f-426c-9bca-de7ada943f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:17:31.522625 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 645/1297] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:17:31 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 16:17:31.547388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb8166a-1f54-4dd2-954d-b67db89d2fc4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:31.653550 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb8166a-1f54-4dd2-954d-b67db89d2fc4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:31.687455 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51289d6-7456-4940-9c8e-c19f1e2ac35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:31.688209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51289d6-7456-4940-9c8e-c19f1e2ac35c None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:31.688494 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51289d6-7456-4940-9c8e-c19f1e2ac35c None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:31.688771 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51289d6-7456-4940-9c8e-c19f1e2ac35c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:31.689424 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51289d6-7456-4940-9c8e-c19f1e2ac35c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:31.690867 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51289d6-7456-4940-9c8e-c19f1e2ac35c None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:31.703264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-d51289d6-7456-4940-9c8e-c19f1e2ac35c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 16:17:31.703264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:31.703264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:17:31.703264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:17:31.703264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:17:31.703264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:17:31.703264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:17:31.703264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:17:31.703264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 16:17:31.703264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:17:31.703264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:17:31.703264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:17:31.703264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:31.703264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:31.703264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:31.703264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:31.703264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:31.704562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:31.704562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:31.704562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:31.704562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:31.704562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:31.704562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:31.704562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:17:31.704562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:17:31.704562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:17:31.704562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:17:31.704562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:31.704562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:31.704562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:31.704562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:31.704562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:31.704562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:31.705839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:31.705839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:31.705839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:31.705839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:31.705839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:31.705839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:31.705839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:31.705839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:31.705839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:31.705839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:31.705839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:17:31.705839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:17:31.705839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 16:17:31.705839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:17:31.705839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 653/1298] 10.222.0.67 () {56 vars in 1117 bytes} [Thu Jun 30 16:17:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:31.710703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41af00-667f-426c-9bca-de7ada943f8c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:31.729519 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:31.810589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a41af00-667f-426c-9bca-de7ada943f8c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:31.845079 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9c455-18c8-43e1-b44c-3316bf8299b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:31.845825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9c455-18c8-43e1-b44c-3316bf8299b5 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:31.846105 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9c455-18c8-43e1-b44c-3316bf8299b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:31.846376 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9c455-18c8-43e1-b44c-3316bf8299b5 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:31.847093 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9c455-18c8-43e1-b44c-3316bf8299b5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:31.847395 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9c455-18c8-43e1-b44c-3316bf8299b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:31.848868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c9c455-18c8-43e1-b44c-3316bf8299b5 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:31.854055 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 646/1299] 10.222.0.67 () {56 vars in 1116 bytes} [Thu Jun 30 16:17:31 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:31.860820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51289d6-7456-4940-9c8e-c19f1e2ac35c None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:31.893372 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:31.962152 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51289d6-7456-4940-9c8e-c19f1e2ac35c None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:31.996747 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e19676-07a6-4718-9421-407837ac1d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:31.997414 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e19676-07a6-4718-9421-407837ac1d36 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:31.998048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e19676-07a6-4718-9421-407837ac1d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:31.998048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e19676-07a6-4718-9421-407837ac1d36 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:31.999043 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e19676-07a6-4718-9421-407837ac1d36 None admin] RBAC: Adding query filter params (interface=public, service_id=829287f02ed1491b89c7b59d09be3f2e, region_id=RegionOne) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:32.000100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e19676-07a6-4718-9421-407837ac1d36 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:32.001525 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e19676-07a6-4718-9421-407837ac1d36 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:32.007034 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 654/1300] 10.222.0.67 () {56 vars in 1251 bytes} [Thu Jun 30 16:17:31 2022] GET /identity/v3/endpoints?service_id=829287f02ed1491b89c7b59d09be3f2e&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:17:32.012610 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9c455-18c8-43e1-b44c-3316bf8299b5 None admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:32.095337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:32.113634 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9c455-18c8-43e1-b44c-3316bf8299b5 None admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:32.148182 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cc63a-8f4a-420b-9d75-af5cca33b23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:32.148838 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cc63a-8f4a-420b-9d75-af5cca33b23f None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:32.149117 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cc63a-8f4a-420b-9d75-af5cca33b23f None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:32.149410 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cc63a-8f4a-420b-9d75-af5cca33b23f None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:32.150060 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009cc63a-8f4a-420b-9d75-af5cca33b23f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:32.151410 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009cc63a-8f4a-420b-9d75-af5cca33b23f None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:32.158489 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 647/1301] 10.222.0.67 () {56 vars in 1087 bytes} [Thu Jun 30 16:17:32 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:17:34.226273 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e501fb3-7995-4d25-9b37-3fea6e97cbe0 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:34.226837 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e501fb3-7995-4d25-9b37-3fea6e97cbe0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:34.226837 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e501fb3-7995-4d25-9b37-3fea6e97cbe0 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:34.228274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 655/1302] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:17:34.233495 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba79c7-1fdf-4916-a36c-1fff93b40870 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:34.233987 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba79c7-1fdf-4916-a36c-1fff93b40870 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:34.234146 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba79c7-1fdf-4916-a36c-1fff93b40870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:34.262164 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7fba79c7-1fdf-4916-a36c-1fff93b40870 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:17:34.262164 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7fba79c7-1fdf-4916-a36c-1fff93b40870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:17:34.319997 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 648/1303] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:17:34 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 16:17:34.344820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e501fb3-7995-4d25-9b37-3fea6e97cbe0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:34.438812 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:34.445472 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e501fb3-7995-4d25-9b37-3fea6e97cbe0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:34.481514 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a957f2e-7a9b-42e8-b6ad-6b55e38cda72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:34.482408 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a957f2e-7a9b-42e8-b6ad-6b55e38cda72 None admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:34.482694 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a957f2e-7a9b-42e8-b6ad-6b55e38cda72 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:34.482970 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a957f2e-7a9b-42e8-b6ad-6b55e38cda72 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:34.483669 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a957f2e-7a9b-42e8-b6ad-6b55e38cda72 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:34.485231 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a957f2e-7a9b-42e8-b6ad-6b55e38cda72 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:34.497051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application [None req-1a957f2e-7a9b-42e8-b6ad-6b55e38cda72 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 16:17:34.497051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:34.497051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 16:17:34.497051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 16:17:34.497051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 16:17:34.497051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 16:17:34.497051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 16:17:34.497051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:17:34.497051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 16:17:34.497051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:17:34.497051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 16:17:34.497051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:17:34.497051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 16:17:34.497051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 16:17:34.497051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 16:17:34.497051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 16:17:34.497051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 16:17:34.498309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 16:17:34.498309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 16:17:34.498309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 16:17:34.498309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 16:17:34.498309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 16:17:34.498309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 16:17:34.498309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 16:17:34.498309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 16:17:34.498309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 16:17:34.498309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 16:17:34.498309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 16:17:34.498309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 16:17:34.498309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 16:17:34.498309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 16:17:34.498309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 16:17:34.498309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 16:17:34.499949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 16:17:34.499949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application with Lock( Jun 30 16:17:34.499949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 16:17:34.499949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self._enter() Jun 30 16:17:34.499949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 16:17:34.499949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 16:17:34.499949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 16:17:34.499949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application return self.creator() Jun 30 16:17:34.499949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 16:17:34.499949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application created_value = creator( Jun 30 16:17:34.499949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 16:17:34.499949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 16:17:34.499949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 16:17:34.499949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: ERROR keystone.server.flask.application Jun 30 16:17:34.499949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 656/1304] 10.222.0.67 () {56 vars in 1117 bytes} [Thu Jun 30 16:17:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 16:17:34.505282 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba79c7-1fdf-4916-a36c-1fff93b40870 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:34.582548 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:34.604736 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba79c7-1fdf-4916-a36c-1fff93b40870 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:34.637851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cefcae-16a8-4c06-be2c-69980ebf7c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:34.638515 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cefcae-16a8-4c06-be2c-69980ebf7c98 None admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:34.638799 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cefcae-16a8-4c06-be2c-69980ebf7c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:34.639077 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cefcae-16a8-4c06-be2c-69980ebf7c98 None admin] PATH_INFO: `/v3/services` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:34.639854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cefcae-16a8-4c06-be2c-69980ebf7c98 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:17:34.640769 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cefcae-16a8-4c06-be2c-69980ebf7c98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:34.642340 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cefcae-16a8-4c06-be2c-69980ebf7c98 None admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:34.647543 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 649/1305] 10.222.0.67 () {56 vars in 1116 bytes} [Thu Jun 30 16:17:34 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:17:34.653309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a957f2e-7a9b-42e8-b6ad-6b55e38cda72 None admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:17:34.753653 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a957f2e-7a9b-42e8-b6ad-6b55e38cda72 None admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:17:34.788315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8b81d-8dad-40f1-bdab-7cfd471154f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:17:34.788971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8b81d-8dad-40f1-bdab-7cfd471154f4 None admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:17:34.789257 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8b81d-8dad-40f1-bdab-7cfd471154f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:17:34.789540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8b81d-8dad-40f1-bdab-7cfd471154f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:17:34.790363 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8b81d-8dad-40f1-bdab-7cfd471154f4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:17:34.791776 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8b81d-8dad-40f1-bdab-7cfd471154f4 None admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:17:34.881514 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 657/1306] 10.222.0.67 () {60 vars in 1144 bytes} [Thu Jun 30 16:17:34 2022] POST /identity/v3/endpoints => generated 331 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:17:44.660824 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=30 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:17:44.876695 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:25:07.723075 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b207190-c7f2-460a-8b7e-a99003252c58 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:07.723667 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b207190-c7f2-460a-8b7e-a99003252c58 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:07.723667 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b207190-c7f2-460a-8b7e-a99003252c58 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:07.725339 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 650/1307] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:25:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:25:07.731391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63826864-27c4-4d1b-8c35-d80c9f0a3faf None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:07.732310 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63826864-27c4-4d1b-8c35-d80c9f0a3faf None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:07.732749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63826864-27c4-4d1b-8c35-d80c9f0a3faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:07.852230 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-63826864-27c4-4d1b-8c35-d80c9f0a3faf None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:25:07.855687 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-63826864-27c4-4d1b-8c35-d80c9f0a3faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:08.033918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 658/1308] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:25:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:25:08.077408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5b74a-6eec-43b8-83f3-13c01bf221eb None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:08.077915 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5b74a-6eec-43b8-83f3-13c01bf221eb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:08.078202 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5b74a-6eec-43b8-83f3-13c01bf221eb None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:08.079617 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 651/1309] 10.222.0.67 () {54 vars in 964 bytes} [Thu Jun 30 16:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:25:08.091305 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc21490-00de-453c-91cd-f5effb082047 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:08.091810 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc21490-00de-453c-91cd-f5effb082047 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:08.092380 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc21490-00de-453c-91cd-f5effb082047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:08.208251 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0fc21490-00de-453c-91cd-f5effb082047 None None] MFA Rules not processed for user `9b508dd8b7244d5f979e1ce7787d2c67`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:25:08.209817 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0fc21490-00de-453c-91cd-f5effb082047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:08.233311 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:25:08.300986 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 659/1310] 10.222.0.67 () {58 vars in 1061 bytes} [Thu Jun 30 16:25:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:25:08.310445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc5b74a-6eec-43b8-83f3-13c01bf221eb None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:25:08.312293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7cc5b74a-6eec-43b8-83f3-13c01bf221eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:08.444374 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc5b74a-6eec-43b8-83f3-13c01bf221eb None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:25:08.543800 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cfe772-4144-4b62-bbce-9d21f1f8cdce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:25:08.544372 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfe772-4144-4b62-bbce-9d21f1f8cdce service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:08.544659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfe772-4144-4b62-bbce-9d21f1f8cdce service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:08.544930 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cfe772-4144-4b62-bbce-9d21f1f8cdce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:08.546619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d2cfe772-4144-4b62-bbce-9d21f1f8cdce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:08.610829 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:25:08.684692 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cfe772-4144-4b62-bbce-9d21f1f8cdce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:25:08.686188 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cfe772-4144-4b62-bbce-9d21f1f8cdce service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:25:08.710318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 652/1311] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:25:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:25:11.054218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8691b3-1635-452f-ad98-493ff63f5d19 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:11.054770 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8691b3-1635-452f-ad98-493ff63f5d19 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:11.054770 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8691b3-1635-452f-ad98-493ff63f5d19 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:11.056390 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 660/1312] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:25:11.061850 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdf0ab-f691-435d-bba7-2858b07adbbc None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:11.062356 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdf0ab-f691-435d-bba7-2858b07adbbc None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:11.062438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdf0ab-f691-435d-bba7-2858b07adbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:11.152459 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a2fdf0ab-f691-435d-bba7-2858b07adbbc None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:25:11.153722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a2fdf0ab-f691-435d-bba7-2858b07adbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:11.252602 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 653/1313] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:25:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:25:11.317529 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360ab2e-e2b9-46bd-af44-840dfda6cd88 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:11.318087 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360ab2e-e2b9-46bd-af44-840dfda6cd88 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:11.318376 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1360ab2e-e2b9-46bd-af44-840dfda6cd88 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:11.320043 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 661/1314] 10.222.0.67 () {54 vars in 964 bytes} [Thu Jun 30 16:25:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:25:11.326837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e2fcc-903e-424c-8497-9ae1fa559bff None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:11.327207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e2fcc-903e-424c-8497-9ae1fa559bff None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:11.327614 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e2fcc-903e-424c-8497-9ae1fa559bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:11.339073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:25:11.411135 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a92e2fcc-903e-424c-8497-9ae1fa559bff None None] MFA Rules not processed for user `9b508dd8b7244d5f979e1ce7787d2c67`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:25:11.412295 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a92e2fcc-903e-424c-8497-9ae1fa559bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:11.492768 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 654/1315] 10.222.0.67 () {58 vars in 1061 bytes} [Thu Jun 30 16:25:11 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:25:11.502337 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360ab2e-e2b9-46bd-af44-840dfda6cd88 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:25:11.504782 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1360ab2e-e2b9-46bd-af44-840dfda6cd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:11.608189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1360ab2e-e2b9-46bd-af44-840dfda6cd88 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:25:11.684538 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:25:11.686905 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54794a-7d34-4e8f-bbe3-7cb8ab7e7e1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:25:11.687584 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54794a-7d34-4e8f-bbe3-7cb8ab7e7e1b service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:11.687874 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54794a-7d34-4e8f-bbe3-7cb8ab7e7e1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:11.688154 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54794a-7d34-4e8f-bbe3-7cb8ab7e7e1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:11.689810 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ef54794a-7d34-4e8f-bbe3-7cb8ab7e7e1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:11.803560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef54794a-7d34-4e8f-bbe3-7cb8ab7e7e1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:25:11.804942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef54794a-7d34-4e8f-bbe3-7cb8ab7e7e1b service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:25:11.825831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 662/1316] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:25:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:25:14.631062 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de07f53-9d1f-492f-b5a0-18dbd5ae279f None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:14.631683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de07f53-9d1f-492f-b5a0-18dbd5ae279f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:14.631683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de07f53-9d1f-492f-b5a0-18dbd5ae279f None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:14.633021 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 655/1317] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:25:14.638609 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bf57b-451d-4a8e-a214-5c001bf8dca1 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:14.639038 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bf57b-451d-4a8e-a214-5c001bf8dca1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:14.639203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bf57b-451d-4a8e-a214-5c001bf8dca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:14.703140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2d4bf57b-451d-4a8e-a214-5c001bf8dca1 None None] MFA Rules not processed for user `3181dca028754df5a6df1154d3635ddd`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:25:14.704389 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2d4bf57b-451d-4a8e-a214-5c001bf8dca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:14.755163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:25:14.799196 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 663/1318] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:25:14 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:25:14.838947 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de07f53-9d1f-492f-b5a0-18dbd5ae279f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:25:14.840585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8de07f53-9d1f-492f-b5a0-18dbd5ae279f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:14.948035 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de07f53-9d1f-492f-b5a0-18dbd5ae279f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:25:14.950824 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:25:15.024283 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cd149b-c02a-4c60-a9e2-eff2c0eec414 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:25:15.024933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd149b-c02a-4c60-a9e2-eff2c0eec414 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:15.025218 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd149b-c02a-4c60-a9e2-eff2c0eec414 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:15.025494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cd149b-c02a-4c60-a9e2-eff2c0eec414 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:15.027156 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c1cd149b-c02a-4c60-a9e2-eff2c0eec414 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:15.135597 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd149b-c02a-4c60-a9e2-eff2c0eec414 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:25:15.137032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cd149b-c02a-4c60-a9e2-eff2c0eec414 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:25:15.156047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 656/1319] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:25:14 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:25:17.988665 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22871679-27af-4c3c-88a0-9060be33b17b None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:17.989237 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22871679-27af-4c3c-88a0-9060be33b17b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:17.989237 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22871679-27af-4c3c-88a0-9060be33b17b None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:17.990632 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 664/1320] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:25:17.996171 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc070882-9490-4d2a-b733-8377e230fccf None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:17.996614 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc070882-9490-4d2a-b733-8377e230fccf None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:17.996709 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc070882-9490-4d2a-b733-8377e230fccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:18.037867 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:25:18.062599 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fc070882-9490-4d2a-b733-8377e230fccf None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:25:18.063861 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fc070882-9490-4d2a-b733-8377e230fccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:18.162598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 657/1321] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:25:17 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:25:18.201486 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22871679-27af-4c3c-88a0-9060be33b17b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:25:18.203022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-22871679-27af-4c3c-88a0-9060be33b17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:18.307328 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22871679-27af-4c3c-88a0-9060be33b17b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:25:18.381814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e7cd3-ff0b-45aa-90ae-843d713b4b9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:25:18.382451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e7cd3-ff0b-45aa-90ae-843d713b4b9c service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:18.382727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e7cd3-ff0b-45aa-90ae-843d713b4b9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:18.382998 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e7cd3-ff0b-45aa-90ae-843d713b4b9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:18.384657 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8a4e7cd3-ff0b-45aa-90ae-843d713b4b9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:18.391915 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:25:18.493464 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e7cd3-ff0b-45aa-90ae-843d713b4b9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:25:18.494911 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e7cd3-ff0b-45aa-90ae-843d713b4b9c service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:25:18.513326 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 665/1322] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:25:18 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:25:21.669192 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64598d9-2de6-4afb-a9c1-90bc2a266908 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:21.669772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64598d9-2de6-4afb-a9c1-90bc2a266908 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:21.669772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64598d9-2de6-4afb-a9c1-90bc2a266908 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:21.671126 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 658/1323] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:25:21.676986 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9e345-408c-4b46-8f26-03e13d80a0d7 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:21.677446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9e345-408c-4b46-8f26-03e13d80a0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:21.677551 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d9e345-408c-4b46-8f26-03e13d80a0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:21.745067 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a7d9e345-408c-4b46-8f26-03e13d80a0d7 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:25:21.746284 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a7d9e345-408c-4b46-8f26-03e13d80a0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:21.803650 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:25:21.842857 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 666/1324] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:25:21.880251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64598d9-2de6-4afb-a9c1-90bc2a266908 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:25:21.881792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a64598d9-2de6-4afb-a9c1-90bc2a266908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:21.999439 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64598d9-2de6-4afb-a9c1-90bc2a266908 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:25:22.014389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:25:22.084629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a6829-98d2-4528-84a7-17e1b36063c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:25:22.085285 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a6829-98d2-4528-84a7-17e1b36063c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:22.085568 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a6829-98d2-4528-84a7-17e1b36063c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:22.085842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a6829-98d2-4528-84a7-17e1b36063c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:22.087688 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b02a6829-98d2-4528-84a7-17e1b36063c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:22.205489 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a6829-98d2-4528-84a7-17e1b36063c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:25:22.207053 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a6829-98d2-4528-84a7-17e1b36063c9 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:25:22.226329 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 659/1325] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:25:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:25:24.330652 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f0994-68fc-40c3-a91f-4e0942856a07 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:24.331211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f0994-68fc-40c3-a91f-4e0942856a07 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:24.331211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f0994-68fc-40c3-a91f-4e0942856a07 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:24.332657 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 667/1326] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:25:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:25:24.338019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f732f1-f964-41bc-bda0-9b13d1bc4fd4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:24.338468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f732f1-f964-41bc-bda0-9b13d1bc4fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:24.338571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f732f1-f964-41bc-bda0-9b13d1bc4fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:24.389748 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:25:24.408581 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d4f732f1-f964-41bc-bda0-9b13d1bc4fd4 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:25:24.410360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d4f732f1-f964-41bc-bda0-9b13d1bc4fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:24.508999 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 660/1327] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:25:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:25:24.547682 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f0994-68fc-40c3-a91f-4e0942856a07 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:25:24.549486 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-118f0994-68fc-40c3-a91f-4e0942856a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:24.666562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f0994-68fc-40c3-a91f-4e0942856a07 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:25:24.742507 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f13326-644a-4747-bbcb-600ebd767cb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:25:24.743158 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f13326-644a-4747-bbcb-600ebd767cb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:24.743441 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f13326-644a-4747-bbcb-600ebd767cb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:24.743746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f13326-644a-4747-bbcb-600ebd767cb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:24.745389 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d9f13326-644a-4747-bbcb-600ebd767cb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:24.762470 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:25:24.856967 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f13326-644a-4747-bbcb-600ebd767cb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:25:24.858430 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f13326-644a-4747-bbcb-600ebd767cb9 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:25:24.881462 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 668/1328] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:25:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:25:26.998382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae661d-f2f4-4ee0-8907-17139e681307 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:26.998797 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae661d-f2f4-4ee0-8907-17139e681307 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:26.998938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae661d-f2f4-4ee0-8907-17139e681307 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:27.000383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 661/1329] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:25:27.006428 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd56a9-82f2-48d6-9bda-fdd72eccf953 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:27.006926 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd56a9-82f2-48d6-9bda-fdd72eccf953 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:27.007016 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd56a9-82f2-48d6-9bda-fdd72eccf953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:27.082344 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-63bd56a9-82f2-48d6-9bda-fdd72eccf953 None None] MFA Rules not processed for user `3181dca028754df5a6df1154d3635ddd`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:25:27.083946 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-63bd56a9-82f2-48d6-9bda-fdd72eccf953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:27.149843 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:25:27.182407 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 669/1330] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:25:27 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:25:27.221999 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae661d-f2f4-4ee0-8907-17139e681307 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:25:27.223594 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-edae661d-f2f4-4ee0-8907-17139e681307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:27.337342 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae661d-f2f4-4ee0-8907-17139e681307 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:25:27.358536 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:25:27.417466 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023f63af-5c02-47ac-9fcd-e007a0acbca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:25:27.418110 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f63af-5c02-47ac-9fcd-e007a0acbca9 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:27.418395 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f63af-5c02-47ac-9fcd-e007a0acbca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:27.418663 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f63af-5c02-47ac-9fcd-e007a0acbca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:27.420328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-023f63af-5c02-47ac-9fcd-e007a0acbca9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:27.537508 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023f63af-5c02-47ac-9fcd-e007a0acbca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:25:27.538971 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023f63af-5c02-47ac-9fcd-e007a0acbca9 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:25:27.558874 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 662/1331] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:25:27 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:25:30.194879 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea47bb-4a44-4509-9bd0-c13bfcb0f6ab None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:30.195408 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea47bb-4a44-4509-9bd0-c13bfcb0f6ab None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:30.195408 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ea47bb-4a44-4509-9bd0-c13bfcb0f6ab None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:30.197023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 670/1332] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:25:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:25:30.202077 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b108a77-db91-4cd5-96e3-7b6fbc1852d5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:30.202557 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b108a77-db91-4cd5-96e3-7b6fbc1852d5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:30.202636 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b108a77-db91-4cd5-96e3-7b6fbc1852d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:30.271688 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:25:30.274146 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8b108a77-db91-4cd5-96e3-7b6fbc1852d5 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:25:30.275429 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8b108a77-db91-4cd5-96e3-7b6fbc1852d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:30.374164 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 663/1333] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:25:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:25:30.411112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea47bb-4a44-4509-9bd0-c13bfcb0f6ab None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:25:30.412831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c3ea47bb-4a44-4509-9bd0-c13bfcb0f6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:30.516209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ea47bb-4a44-4509-9bd0-c13bfcb0f6ab None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:25:30.589860 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe74ea1-e8e5-4fb7-8adf-e8b7b9bd84db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:25:30.590514 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe74ea1-e8e5-4fb7-8adf-e8b7b9bd84db service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:30.590793 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe74ea1-e8e5-4fb7-8adf-e8b7b9bd84db service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:30.591074 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe74ea1-e8e5-4fb7-8adf-e8b7b9bd84db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:30.592758 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6fe74ea1-e8e5-4fb7-8adf-e8b7b9bd84db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:30.619612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:25:30.701806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe74ea1-e8e5-4fb7-8adf-e8b7b9bd84db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:25:30.703267 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe74ea1-e8e5-4fb7-8adf-e8b7b9bd84db service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:25:30.721516 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 671/1334] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:25:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:25:32.822369 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88678c-c446-496d-9293-12d7abbc8799 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:32.822939 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88678c-c446-496d-9293-12d7abbc8799 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:32.822939 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e88678c-c446-496d-9293-12d7abbc8799 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:32.824311 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 664/1335] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:25:32.829505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def1db9-7f14-42e0-b8a2-57bc581c7aed None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:32.829925 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def1db9-7f14-42e0-b8a2-57bc581c7aed None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:32.830037 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def1db9-7f14-42e0-b8a2-57bc581c7aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:32.895425 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9def1db9-7f14-42e0-b8a2-57bc581c7aed None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:25:32.896662 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9def1db9-7f14-42e0-b8a2-57bc581c7aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:32.972373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:25:32.995087 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 672/1336] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:25:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:25:33.036473 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88678c-c446-496d-9293-12d7abbc8799 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:25:33.037999 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5e88678c-c446-496d-9293-12d7abbc8799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:33.145167 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e88678c-c446-496d-9293-12d7abbc8799 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:25:33.174201 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:25:33.221711 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7f32e6-e4cf-4474-b90f-8ffc51b6d010 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:25:33.222571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f32e6-e4cf-4474-b90f-8ffc51b6d010 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:33.222809 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f32e6-e4cf-4474-b90f-8ffc51b6d010 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:33.223089 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7f32e6-e4cf-4474-b90f-8ffc51b6d010 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:33.224781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0b7f32e6-e4cf-4474-b90f-8ffc51b6d010 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:33.332414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f32e6-e4cf-4474-b90f-8ffc51b6d010 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:25:33.333846 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7f32e6-e4cf-4474-b90f-8ffc51b6d010 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:25:33.352138 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 665/1337] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:25:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:25:36.204169 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2145b57-2f35-45d7-8f7d-6bf763523d5a None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:36.204707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2145b57-2f35-45d7-8f7d-6bf763523d5a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:36.204707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2145b57-2f35-45d7-8f7d-6bf763523d5a None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:36.206592 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 673/1338] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:25:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:25:36.212240 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303dcd7-9bef-46a0-bd04-3b03983f0d77 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:36.212699 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303dcd7-9bef-46a0-bd04-3b03983f0d77 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:36.212885 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303dcd7-9bef-46a0-bd04-3b03983f0d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:36.277330 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8303dcd7-9bef-46a0-bd04-3b03983f0d77 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:25:36.278451 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8303dcd7-9bef-46a0-bd04-3b03983f0d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:36.286846 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:25:36.373689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 666/1339] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:25:36 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:25:36.410354 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2145b57-2f35-45d7-8f7d-6bf763523d5a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:25:36.411945 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c2145b57-2f35-45d7-8f7d-6bf763523d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:36.513988 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2145b57-2f35-45d7-8f7d-6bf763523d5a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:25:36.587678 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41feb4-57d6-44e5-8a58-f4c37cccfd3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:25:36.588324 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41feb4-57d6-44e5-8a58-f4c37cccfd3b service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:36.588611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41feb4-57d6-44e5-8a58-f4c37cccfd3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:36.588912 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41feb4-57d6-44e5-8a58-f4c37cccfd3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:36.590555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ad41feb4-57d6-44e5-8a58-f4c37cccfd3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:36.625110 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:25:36.699175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41feb4-57d6-44e5-8a58-f4c37cccfd3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:25:36.700647 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad41feb4-57d6-44e5-8a58-f4c37cccfd3b service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:25:36.718977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 674/1340] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:25:36 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:25:40.993566 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b92df-9f25-41fb-90a6-0cc3331c8e34 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:40.994098 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b92df-9f25-41fb-90a6-0cc3331c8e34 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:40.994098 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b92df-9f25-41fb-90a6-0cc3331c8e34 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:40.995688 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 667/1341] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:25:41.001235 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13a86e-1d17-4d28-a888-18cfffcc4978 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:41.001667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13a86e-1d17-4d28-a888-18cfffcc4978 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:41.001840 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de13a86e-1d17-4d28-a888-18cfffcc4978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:41.090483 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-de13a86e-1d17-4d28-a888-18cfffcc4978 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:25:41.091736 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-de13a86e-1d17-4d28-a888-18cfffcc4978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:41.201700 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:25:41.218275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 675/1342] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:25:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:25:41.258029 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b92df-9f25-41fb-90a6-0cc3331c8e34 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:25:41.260235 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a36b92df-9f25-41fb-90a6-0cc3331c8e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:41.359781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b92df-9f25-41fb-90a6-0cc3331c8e34 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:25:41.363904 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a174de-7dc6-4378-8f2d-ae6b673e0506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:25:41.364550 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a174de-7dc6-4378-8f2d-ae6b673e0506 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:41.364830 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a174de-7dc6-4378-8f2d-ae6b673e0506 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:41.365103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a174de-7dc6-4378-8f2d-ae6b673e0506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:41.374152 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a174de-7dc6-4378-8f2d-ae6b673e0506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:25:41.375602 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a174de-7dc6-4378-8f2d-ae6b673e0506 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:25:41.385965 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 668/1343] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:25:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:25:45.027962 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73909e-f813-417f-b4c7-a292ef8d3584 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:45.028514 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73909e-f813-417f-b4c7-a292ef8d3584 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:45.028514 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d73909e-f813-417f-b4c7-a292ef8d3584 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:45.030400 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 676/1344] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:25:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:25:45.035772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8dea5-2d08-40a0-9cc4-ae9681b406f9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:45.036193 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8dea5-2d08-40a0-9cc4-ae9681b406f9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:45.036342 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8dea5-2d08-40a0-9cc4-ae9681b406f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:45.066499 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-89f8dea5-2d08-40a0-9cc4-ae9681b406f9 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:25:45.072294 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-89f8dea5-2d08-40a0-9cc4-ae9681b406f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:45.096862 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:25:45.147534 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 669/1345] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:25:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:25:45.185537 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73909e-f813-417f-b4c7-a292ef8d3584 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:25:45.187270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1d73909e-f813-417f-b4c7-a292ef8d3584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:45.201072 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d73909e-f813-417f-b4c7-a292ef8d3584 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:25:45.204559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2922956e-b73f-4d61-8643-3635524a5da7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:25:45.205202 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922956e-b73f-4d61-8643-3635524a5da7 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:45.205543 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922956e-b73f-4d61-8643-3635524a5da7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:45.205842 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922956e-b73f-4d61-8643-3635524a5da7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:45.215326 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2922956e-b73f-4d61-8643-3635524a5da7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:25:45.216842 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2922956e-b73f-4d61-8643-3635524a5da7 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:25:45.226810 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 677/1346] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:25:45 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:25:49.998535 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4799a-b631-490f-8d59-79e8af5e50ec None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:49.998908 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4799a-b631-490f-8d59-79e8af5e50ec None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:49.998908 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4799a-b631-490f-8d59-79e8af5e50ec None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:50.000328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 670/1347] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:25:50.006539 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0fb8c-5bef-4325-97b9-8991f4f3bb54 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:50.007022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0fb8c-5bef-4325-97b9-8991f4f3bb54 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:50.007104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0fb8c-5bef-4325-97b9-8991f4f3bb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:50.025611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f5d0fb8c-5bef-4325-97b9-8991f4f3bb54 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:25:50.026826 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f5d0fb8c-5bef-4325-97b9-8991f4f3bb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:50.035797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 678/1348] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:25:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:25:50.076389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4799a-b631-490f-8d59-79e8af5e50ec None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:25:50.088272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4799a-b631-490f-8d59-79e8af5e50ec None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:25:50.091844 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb90396-676d-4528-ba56-20686ff59b50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:25:50.092497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb90396-676d-4528-ba56-20686ff59b50 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:50.092780 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb90396-676d-4528-ba56-20686ff59b50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:50.093056 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb90396-676d-4528-ba56-20686ff59b50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:50.103378 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb90396-676d-4528-ba56-20686ff59b50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:25:50.104920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb90396-676d-4528-ba56-20686ff59b50 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:25:50.115600 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 671/1349] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:25:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:25:55.118425 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1350a64-319f-4780-ba2d-6cc5d2312744 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:55.119290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1350a64-319f-4780-ba2d-6cc5d2312744 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:55.119767 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1350a64-319f-4780-ba2d-6cc5d2312744 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:55.122631 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 679/1350] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 30 16:25:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:25:55.129140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58686954-95e6-4bfd-a340-3f399c0c07b3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:55.129637 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58686954-95e6-4bfd-a340-3f399c0c07b3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:55.129952 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58686954-95e6-4bfd-a340-3f399c0c07b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:55.229254 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-58686954-95e6-4bfd-a340-3f399c0c07b3 None None] MFA Rules not processed for user `a932a9390fdd47f3adcf053a60250944`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:25:55.230715 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-58686954-95e6-4bfd-a340-3f399c0c07b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:55.349857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 672/1351] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 30 16:25:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:25:59.700938 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbf594-bab0-4798-b57c-1e9bd83c6b06 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:59.701934 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbf594-bab0-4798-b57c-1e9bd83c6b06 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:59.702219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbf594-bab0-4798-b57c-1e9bd83c6b06 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:59.703842 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 680/1352] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 30 16:25:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:25:59.709164 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8789d2-d49a-428e-9929-d9c5d0db3f32 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:25:59.710007 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8789d2-d49a-428e-9929-d9c5d0db3f32 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:25:59.710371 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8789d2-d49a-428e-9929-d9c5d0db3f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:25:59.729710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2a8789d2-d49a-428e-9929-d9c5d0db3f32 None None] MFA Rules not processed for user `a932a9390fdd47f3adcf053a60250944`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:25:59.731015 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2a8789d2-d49a-428e-9929-d9c5d0db3f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:25:59.741452 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 673/1353] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 30 16:25:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:26:00.020373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=13 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:26:01.321114 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855221c-5af5-466b-b0f9-4dfbe30048d6 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:01.321734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855221c-5af5-466b-b0f9-4dfbe30048d6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:01.321734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855221c-5af5-466b-b0f9-4dfbe30048d6 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:01.323318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 681/1354] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 30 16:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:26:01.328540 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d764c6a-f189-4a7e-ac0a-debe79899d89 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:01.328945 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d764c6a-f189-4a7e-ac0a-debe79899d89 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:01.329104 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d764c6a-f189-4a7e-ac0a-debe79899d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:01.347956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5d764c6a-f189-4a7e-ac0a-debe79899d89 None None] MFA Rules not processed for user `a932a9390fdd47f3adcf053a60250944`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:26:01.349158 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5d764c6a-f189-4a7e-ac0a-debe79899d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:26:01.357981 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 674/1355] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 30 16:26:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:26:04.964409 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7303a-81ac-4103-905e-96acb5db6e39 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:04.964962 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7303a-81ac-4103-905e-96acb5db6e39 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:04.964962 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e7303a-81ac-4103-905e-96acb5db6e39 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:04.970295 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 682/1356] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:26:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:26:04.972301 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f5e52-adf5-43cb-aaa3-98b1d6b767d0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:04.973174 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f5e52-adf5-43cb-aaa3-98b1d6b767d0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:04.973774 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f5e52-adf5-43cb-aaa3-98b1d6b767d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:04.994732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bc8f5e52-adf5-43cb-aaa3-98b1d6b767d0 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:26:04.996893 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bc8f5e52-adf5-43cb-aaa3-98b1d6b767d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:26:05.008512 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 675/1357] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:26:05.066370 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c83a0-ba01-4898-961a-6c8cdf72d78d None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:05.066659 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c83a0-ba01-4898-961a-6c8cdf72d78d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:05.066940 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c83a0-ba01-4898-961a-6c8cdf72d78d None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:05.068659 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 683/1358] 10.222.0.67 () {54 vars in 961 bytes} [Thu Jun 30 16:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:26:05.078550 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35601034-f6b5-4e23-bfa7-33657b5c4c18 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:05.079204 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35601034-f6b5-4e23-bfa7-33657b5c4c18 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:05.080269 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35601034-f6b5-4e23-bfa7-33657b5c4c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:05.128232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:26:05.137751 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-35601034-f6b5-4e23-bfa7-33657b5c4c18 None None] MFA Rules not processed for user `71cca3708386459197abf6c67505bc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:26:05.139000 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-35601034-f6b5-4e23-bfa7-33657b5c4c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:26:05.227851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 676/1359] 10.222.0.67 () {58 vars in 1058 bytes} [Thu Jun 30 16:26:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:26:05.236691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c83a0-ba01-4898-961a-6c8cdf72d78d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:26:05.251374 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c83a0-ba01-4898-961a-6c8cdf72d78d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:26:05.257354 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3270ca2-ff85-49b7-83d9-0b71d5981339 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:26:05.258020 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3270ca2-ff85-49b7-83d9-0b71d5981339 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:05.258299 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3270ca2-ff85-49b7-83d9-0b71d5981339 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:05.258573 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3270ca2-ff85-49b7-83d9-0b71d5981339 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:05.267371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3270ca2-ff85-49b7-83d9-0b71d5981339 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:26:05.269028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3270ca2-ff85-49b7-83d9-0b71d5981339 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:26:05.279331 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 684/1360] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:26:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:26:07.557877 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58122e9f-8f13-4f42-9c76-a47f17675a90 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:07.558538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58122e9f-8f13-4f42-9c76-a47f17675a90 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:07.558538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58122e9f-8f13-4f42-9c76-a47f17675a90 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:07.559746 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 677/1361] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:26:07.565941 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820aa1d-76ce-4ea1-befc-3b1869e2c07d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:07.566431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820aa1d-76ce-4ea1-befc-3b1869e2c07d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:07.566543 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820aa1d-76ce-4ea1-befc-3b1869e2c07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:07.584742 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0820aa1d-76ce-4ea1-befc-3b1869e2c07d None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:26:07.585927 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0820aa1d-76ce-4ea1-befc-3b1869e2c07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:26:07.593928 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 685/1362] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:26:07.640346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58122e9f-8f13-4f42-9c76-a47f17675a90 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:26:07.656751 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58122e9f-8f13-4f42-9c76-a47f17675a90 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:26:07.663750 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef3fa6-35cb-4161-b0d5-bcf08f7cea1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:26:07.664499 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef3fa6-35cb-4161-b0d5-bcf08f7cea1c service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:07.664880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef3fa6-35cb-4161-b0d5-bcf08f7cea1c service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:07.665222 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef3fa6-35cb-4161-b0d5-bcf08f7cea1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:07.675023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef3fa6-35cb-4161-b0d5-bcf08f7cea1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:26:07.676670 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef3fa6-35cb-4161-b0d5-bcf08f7cea1c service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:26:07.686706 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 678/1363] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:26:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:26:10.271848 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03730e32-d505-42e2-ab1c-752cf7b62fb2 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:10.272342 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03730e32-d505-42e2-ab1c-752cf7b62fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:10.272586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03730e32-d505-42e2-ab1c-752cf7b62fb2 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:10.274348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 686/1364] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:26:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:26:10.280304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347d70e-79e2-49af-8c9b-d60c75022ee1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:10.280832 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347d70e-79e2-49af-8c9b-d60c75022ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:10.281104 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347d70e-79e2-49af-8c9b-d60c75022ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:10.305507 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1347d70e-79e2-49af-8c9b-d60c75022ee1 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:26:10.306725 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1347d70e-79e2-49af-8c9b-d60c75022ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:26:10.315626 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 679/1365] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:26:10.362541 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8faf194-7c92-4e8a-961c-5ce3c4103b77 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:10.362933 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8faf194-7c92-4e8a-961c-5ce3c4103b77 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:10.363425 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8faf194-7c92-4e8a-961c-5ce3c4103b77 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:10.365013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 687/1366] 10.222.0.67 () {54 vars in 961 bytes} [Thu Jun 30 16:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:26:10.370639 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54586b9b-7f8c-4420-a463-c528db86daeb None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:10.370932 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54586b9b-7f8c-4420-a463-c528db86daeb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:10.373849 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54586b9b-7f8c-4420-a463-c528db86daeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:10.390477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-54586b9b-7f8c-4420-a463-c528db86daeb None None] MFA Rules not processed for user `71cca3708386459197abf6c67505bc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:26:10.392333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-54586b9b-7f8c-4420-a463-c528db86daeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:26:10.402582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 680/1367] 10.222.0.67 () {58 vars in 1058 bytes} [Thu Jun 30 16:26:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:26:10.412704 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8faf194-7c92-4e8a-961c-5ce3c4103b77 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:26:10.427764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8faf194-7c92-4e8a-961c-5ce3c4103b77 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:26:10.433880 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c843e01-3049-4839-abf7-973aa15feb4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:26:10.434532 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c843e01-3049-4839-abf7-973aa15feb4c service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:10.434822 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c843e01-3049-4839-abf7-973aa15feb4c service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:10.435099 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c843e01-3049-4839-abf7-973aa15feb4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:10.444021 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c843e01-3049-4839-abf7-973aa15feb4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:26:10.445621 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c843e01-3049-4839-abf7-973aa15feb4c service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:26:10.456702 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 688/1368] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:26:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:26:12.789092 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676368fb-be5f-49b9-bb4a-58a6a4dd3fc7 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:12.789629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676368fb-be5f-49b9-bb4a-58a6a4dd3fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:12.789629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676368fb-be5f-49b9-bb4a-58a6a4dd3fc7 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:12.791164 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 681/1369] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:26:12.796321 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281aec0-4af9-4aa8-bed0-ddbc2e3896f6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:12.797218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281aec0-4af9-4aa8-bed0-ddbc2e3896f6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:12.797505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281aec0-4af9-4aa8-bed0-ddbc2e3896f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:12.816684 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d281aec0-4af9-4aa8-bed0-ddbc2e3896f6 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:26:12.817884 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d281aec0-4af9-4aa8-bed0-ddbc2e3896f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:26:12.825971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 689/1370] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:26:12.871992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676368fb-be5f-49b9-bb4a-58a6a4dd3fc7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:26:12.887588 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676368fb-be5f-49b9-bb4a-58a6a4dd3fc7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:26:12.893570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1bd93-9df5-46b7-8b26-b96e5bc68fcb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:26:12.894228 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1bd93-9df5-46b7-8b26-b96e5bc68fcb service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:12.894493 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1bd93-9df5-46b7-8b26-b96e5bc68fcb service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:12.894762 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1bd93-9df5-46b7-8b26-b96e5bc68fcb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:12.903242 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1bd93-9df5-46b7-8b26-b96e5bc68fcb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:26:12.905097 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1bd93-9df5-46b7-8b26-b96e5bc68fcb service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:26:12.915985 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 682/1371] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:26:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:26:13.332617 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ca15b-5aa9-4b57-9c4e-d7a9ee827b21 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:13.333172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ca15b-5aa9-4b57-9c4e-d7a9ee827b21 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:13.333621 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ca15b-5aa9-4b57-9c4e-d7a9ee827b21 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:13.335138 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 690/1372] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:26:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:26:13.340627 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943b994-da43-4d4b-af26-871c473f83b2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:13.340920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943b994-da43-4d4b-af26-871c473f83b2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:13.341200 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4943b994-da43-4d4b-af26-871c473f83b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:13.360084 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4943b994-da43-4d4b-af26-871c473f83b2 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:26:13.361841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4943b994-da43-4d4b-af26-871c473f83b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:26:13.369874 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 683/1373] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:26:13.718785 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6d76a-5e8f-482f-ba0c-bca2031b5151 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:13.718785 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6d76a-5e8f-482f-ba0c-bca2031b5151 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:13.718785 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6d76a-5e8f-482f-ba0c-bca2031b5151 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:13.718785 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 691/1374] 10.222.0.67 () {54 vars in 963 bytes} [Thu Jun 30 16:26:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:26:13.725203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d15d45-5d16-4f84-bbf0-876dd92fb650 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:13.725203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d15d45-5d16-4f84-bbf0-876dd92fb650 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:13.725203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d15d45-5d16-4f84-bbf0-876dd92fb650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:13.778594 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-55d15d45-5d16-4f84-bbf0-876dd92fb650 None None] MFA Rules not processed for user `61437a2fa2a94b35b1b4a5750f6e2cad`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:26:13.780210 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-55d15d45-5d16-4f84-bbf0-876dd92fb650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:26:13.849344 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 684/1375] 10.222.0.67 () {58 vars in 1060 bytes} [Thu Jun 30 16:26:13 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:26:13.857396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af6d76a-5e8f-482f-ba0c-bca2031b5151 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:26:13.870707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af6d76a-5e8f-482f-ba0c-bca2031b5151 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:26:13.874972 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca4bfdc-9c6e-4d34-a7e0-dc3a465b5753 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:26:13.875731 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4bfdc-9c6e-4d34-a7e0-dc3a465b5753 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:13.878143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4bfdc-9c6e-4d34-a7e0-dc3a465b5753 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:13.878143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4bfdc-9c6e-4d34-a7e0-dc3a465b5753 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:13.887573 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca4bfdc-9c6e-4d34-a7e0-dc3a465b5753 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:26:13.889115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca4bfdc-9c6e-4d34-a7e0-dc3a465b5753 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:26:13.899915 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 692/1376] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:26:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:26:15.244514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6025a67-1eb6-4f7f-b8cc-61b8cab1147c None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:15.245118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6025a67-1eb6-4f7f-b8cc-61b8cab1147c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:15.245118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6025a67-1eb6-4f7f-b8cc-61b8cab1147c None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:15.246433 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 685/1377] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:26:15.252459 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d952711-23a7-4bba-8cba-b63ba3541a4e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:15.252948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d952711-23a7-4bba-8cba-b63ba3541a4e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:15.252948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d952711-23a7-4bba-8cba-b63ba3541a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:15.271010 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9d952711-23a7-4bba-8cba-b63ba3541a4e None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:26:15.272232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9d952711-23a7-4bba-8cba-b63ba3541a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:26:15.280649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 693/1378] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:26:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:26:15.326468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6025a67-1eb6-4f7f-b8cc-61b8cab1147c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:26:15.341686 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6025a67-1eb6-4f7f-b8cc-61b8cab1147c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:26:15.348029 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd9be90-ee15-4c81-9c6f-65b33b7122fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:26:15.348694 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9be90-ee15-4c81-9c6f-65b33b7122fd service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:15.348976 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9be90-ee15-4c81-9c6f-65b33b7122fd service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:15.349252 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9be90-ee15-4c81-9c6f-65b33b7122fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:15.358051 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd9be90-ee15-4c81-9c6f-65b33b7122fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:26:15.359663 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd9be90-ee15-4c81-9c6f-65b33b7122fd service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:26:15.364474 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:26:15.370857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 686/1379] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:26:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:26:17.688251 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e46a3-e9de-4b80-805a-d6d5304bbc0c None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:17.688806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e46a3-e9de-4b80-805a-d6d5304bbc0c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:17.689004 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e46a3-e9de-4b80-805a-d6d5304bbc0c None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:17.691003 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 694/1380] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:26:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:26:17.696785 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712255d-b22b-40a2-9e66-05831e3dacd5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:17.697247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712255d-b22b-40a2-9e66-05831e3dacd5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:17.697343 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e712255d-b22b-40a2-9e66-05831e3dacd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:17.716492 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e712255d-b22b-40a2-9e66-05831e3dacd5 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:26:17.717605 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e712255d-b22b-40a2-9e66-05831e3dacd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:26:17.726010 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 687/1381] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:26:17.772569 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e46a3-e9de-4b80-805a-d6d5304bbc0c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:26:17.788499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e46a3-e9de-4b80-805a-d6d5304bbc0c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:26:17.794871 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c1c223-1f37-4ef1-9ea4-d9bb9094d24c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:26:17.795554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c1c223-1f37-4ef1-9ea4-d9bb9094d24c service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:17.795845 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c1c223-1f37-4ef1-9ea4-d9bb9094d24c service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:17.796127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c1c223-1f37-4ef1-9ea4-d9bb9094d24c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:17.805418 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c1c223-1f37-4ef1-9ea4-d9bb9094d24c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:26:17.808765 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c1c223-1f37-4ef1-9ea4-d9bb9094d24c service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:26:17.819966 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 695/1382] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:26:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:26:20.048244 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83990c-f73b-439b-8071-0ba970d0f2ff None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:20.048829 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83990c-f73b-439b-8071-0ba970d0f2ff None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:20.049015 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83990c-f73b-439b-8071-0ba970d0f2ff None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:20.050296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 688/1383] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:26:20.055906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b78ff-52d1-4a91-8dc2-4e14d8809c8e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:20.056328 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b78ff-52d1-4a91-8dc2-4e14d8809c8e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:20.056664 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b78ff-52d1-4a91-8dc2-4e14d8809c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:20.075821 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-170b78ff-52d1-4a91-8dc2-4e14d8809c8e None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:26:20.077142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-170b78ff-52d1-4a91-8dc2-4e14d8809c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:26:20.086608 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 696/1384] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:26:20.130021 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83990c-f73b-439b-8071-0ba970d0f2ff None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:26:20.144577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83990c-f73b-439b-8071-0ba970d0f2ff None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:26:20.150552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112660c9-e388-4550-aefa-531377a050bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:26:20.151123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112660c9-e388-4550-aefa-531377a050bd service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:20.151402 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112660c9-e388-4550-aefa-531377a050bd service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:20.151790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112660c9-e388-4550-aefa-531377a050bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:20.160358 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112660c9-e388-4550-aefa-531377a050bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:26:20.161912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112660c9-e388-4550-aefa-531377a050bd service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:26:20.171296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 689/1385] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:26:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:26:22.380912 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd256e96-dd15-4bad-afe9-e7e4df86113c None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:22.381501 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd256e96-dd15-4bad-afe9-e7e4df86113c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:22.381501 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd256e96-dd15-4bad-afe9-e7e4df86113c None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:22.382960 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 697/1386] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:26:22.389548 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9be6f-fffb-4661-b7eb-64735ab9862f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:22.390010 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9be6f-fffb-4661-b7eb-64735ab9862f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:22.390117 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9be6f-fffb-4661-b7eb-64735ab9862f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:22.409734 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-03d9be6f-fffb-4661-b7eb-64735ab9862f None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:26:22.411007 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-03d9be6f-fffb-4661-b7eb-64735ab9862f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:26:22.419331 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 690/1387] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:26:22.478634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd256e96-dd15-4bad-afe9-e7e4df86113c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:26:22.494702 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd256e96-dd15-4bad-afe9-e7e4df86113c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:26:22.500444 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97174328-3841-4baf-8a9f-ced0b06bf28b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:26:22.501120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97174328-3841-4baf-8a9f-ced0b06bf28b service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:22.501426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97174328-3841-4baf-8a9f-ced0b06bf28b service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:22.502308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97174328-3841-4baf-8a9f-ced0b06bf28b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:22.511676 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97174328-3841-4baf-8a9f-ced0b06bf28b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:26:22.513352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97174328-3841-4baf-8a9f-ced0b06bf28b service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:26:22.524220 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 698/1388] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:26:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:26:24.843495 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b25d20-ae86-475c-ab4e-61e343b2640a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:24.844197 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b25d20-ae86-475c-ab4e-61e343b2640a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:24.844197 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b25d20-ae86-475c-ab4e-61e343b2640a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:24.845561 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 691/1389] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:26:24.851155 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928b739-024a-4e52-913e-cac0d7e6999e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:24.851800 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928b739-024a-4e52-913e-cac0d7e6999e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:24.851800 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928b739-024a-4e52-913e-cac0d7e6999e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:24.872336 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9928b739-024a-4e52-913e-cac0d7e6999e None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:26:24.873576 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9928b739-024a-4e52-913e-cac0d7e6999e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:26:24.882661 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 699/1390] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:26:24.933167 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b25d20-ae86-475c-ab4e-61e343b2640a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:26:24.948701 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b25d20-ae86-475c-ab4e-61e343b2640a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:26:24.954805 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398ce81-8b99-4962-b617-8127fd035931 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:26:24.955457 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398ce81-8b99-4962-b617-8127fd035931 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:24.955771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398ce81-8b99-4962-b617-8127fd035931 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:24.956043 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398ce81-8b99-4962-b617-8127fd035931 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:24.965336 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398ce81-8b99-4962-b617-8127fd035931 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:26:24.966922 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6398ce81-8b99-4962-b617-8127fd035931 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:26:24.977713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 692/1391] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:26:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:26:27.186414 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d70a4d-1702-4ddd-87dd-b57302dc6da2 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:27.186923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d70a4d-1702-4ddd-87dd-b57302dc6da2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:27.186923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d70a4d-1702-4ddd-87dd-b57302dc6da2 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:27.188361 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 700/1392] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:26:27.194673 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef2196-6309-452c-9daa-54670685730b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:27.195160 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef2196-6309-452c-9daa-54670685730b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:27.195402 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef2196-6309-452c-9daa-54670685730b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:27.221213 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-59ef2196-6309-452c-9daa-54670685730b None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:26:27.222565 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-59ef2196-6309-452c-9daa-54670685730b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:26:27.232640 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 693/1393] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:26:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:26:27.279789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d70a4d-1702-4ddd-87dd-b57302dc6da2 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:26:27.297667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d70a4d-1702-4ddd-87dd-b57302dc6da2 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:26:27.303909 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f79313-b850-4296-a2bc-038073dd93af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:26:27.304579 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f79313-b850-4296-a2bc-038073dd93af service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:27.304879 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f79313-b850-4296-a2bc-038073dd93af service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:27.305173 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f79313-b850-4296-a2bc-038073dd93af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:27.314331 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f79313-b850-4296-a2bc-038073dd93af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:26:27.316043 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f79313-b850-4296-a2bc-038073dd93af service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:26:27.327194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 701/1394] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:26:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:26:29.440060 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdbbc1-f16d-4a01-b772-95741ff28843 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:29.440631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdbbc1-f16d-4a01-b772-95741ff28843 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:29.440631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fdbbc1-f16d-4a01-b772-95741ff28843 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:29.442049 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 694/1395] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:26:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:26:29.447803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af2ef6-2b22-40d4-837b-225a19a02e19 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:29.448252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af2ef6-2b22-40d4-837b-225a19a02e19 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:29.448374 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af2ef6-2b22-40d4-837b-225a19a02e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:29.467214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d3af2ef6-2b22-40d4-837b-225a19a02e19 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:26:29.468448 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d3af2ef6-2b22-40d4-837b-225a19a02e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:26:29.477903 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 702/1396] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:26:29.535967 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06617780-e93d-440d-997c-fe7544d16339 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:29.536251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06617780-e93d-440d-997c-fe7544d16339 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:29.536525 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06617780-e93d-440d-997c-fe7544d16339 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:29.537915 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 695/1397] 10.222.0.67 () {54 vars in 962 bytes} [Thu Jun 30 16:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:26:29.542922 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4a127-eaeb-4e98-8a32-cd2cb8c13462 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:29.543210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4a127-eaeb-4e98-8a32-cd2cb8c13462 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:29.543517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4a127-eaeb-4e98-8a32-cd2cb8c13462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:29.567209 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c755b5-90db-4131-a54d-8cd7f1f8697a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:29.567523 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c755b5-90db-4131-a54d-8cd7f1f8697a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:29.567810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c755b5-90db-4131-a54d-8cd7f1f8697a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:29.569223 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 696/1398] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:26:29.574415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecdf3c-71ab-4f04-8f1b-6e4d922fdfa6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:29.574699 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecdf3c-71ab-4f04-8f1b-6e4d922fdfa6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:29.575010 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eecdf3c-71ab-4f04-8f1b-6e4d922fdfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:29.592807 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6eecdf3c-71ab-4f04-8f1b-6e4d922fdfa6 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:26:29.593335 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-49d4a127-eaeb-4e98-8a32-cd2cb8c13462 None None] MFA Rules not processed for user `ec41c84c9375437590c42621ca5b400a`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:26:29.594014 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6eecdf3c-71ab-4f04-8f1b-6e4d922fdfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:26:29.594505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-49d4a127-eaeb-4e98-8a32-cd2cb8c13462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:26:29.601892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 697/1399] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:26:29.648181 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecdf3c-71ab-4f04-8f1b-6e4d922fdfa6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:26:29.661032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eecdf3c-71ab-4f04-8f1b-6e4d922fdfa6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:26:29.663190 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 703/1400] 10.222.0.67 () {58 vars in 1059 bytes} [Thu Jun 30 16:26:29 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:26:29.666709 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e3274-5163-4a82-90b7-2878965b6862 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:26:29.667359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e3274-5163-4a82-90b7-2878965b6862 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:29.667677 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e3274-5163-4a82-90b7-2878965b6862 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:29.667954 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e3274-5163-4a82-90b7-2878965b6862 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:29.670632 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4a127-eaeb-4e98-8a32-cd2cb8c13462 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:26:29.678155 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e3274-5163-4a82-90b7-2878965b6862 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:26:29.678910 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4a127-eaeb-4e98-8a32-cd2cb8c13462 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:26:29.679769 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e3274-5163-4a82-90b7-2878965b6862 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:26:29.682577 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238c9a2-4c45-4ec4-b99d-54c50786a0b9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec41c84c9375437590c42621ca5b400a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:26:29.683207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238c9a2-4c45-4ec4-b99d-54c50786a0b9 service glance] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:29.683513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238c9a2-4c45-4ec4-b99d-54c50786a0b9 service glance] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:29.683791 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238c9a2-4c45-4ec4-b99d-54c50786a0b9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:29.689745 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 698/1401] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:26:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:26:29.693006 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6238c9a2-4c45-4ec4-b99d-54c50786a0b9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:26:29.694385 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6238c9a2-4c45-4ec4-b99d-54c50786a0b9 service glance] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:26:29.704149 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 704/1402] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:26:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:26:30.675015 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12f463-d28b-4ba8-adbc-f584a2783c8c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:30.675661 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12f463-d28b-4ba8-adbc-f584a2783c8c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:30.675779 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12f463-d28b-4ba8-adbc-f584a2783c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:30.730488 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4f12f463-d28b-4ba8-adbc-f584a2783c8c None None] MFA Rules not processed for user `d49604bffa3d4e89a631eefa6d4e4f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:26:30.731729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4f12f463-d28b-4ba8-adbc-f584a2783c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:26:30.801092 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 699/1403] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 30 16:26:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:26:30.843895 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba2152-109f-432d-ad51-aa8e4ad4a83c None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:30.844416 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba2152-109f-432d-ad51-aa8e4ad4a83c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:30.844537 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba2152-109f-432d-ad51-aa8e4ad4a83c None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:30.845923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 705/1404] 10.222.0.67 () {54 vars in 942 bytes} [Thu Jun 30 16:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:26:30.852949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0d0a8-ff5b-491f-b47b-f507f0e1fcf1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:30.853318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0d0a8-ff5b-491f-b47b-f507f0e1fcf1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:30.853674 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0d0a8-ff5b-491f-b47b-f507f0e1fcf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:30.906555 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a5e0d0a8-ff5b-491f-b47b-f507f0e1fcf1 None None] MFA Rules not processed for user `988b917eea4b48ffae32e23b8555a5c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:26:30.908060 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a5e0d0a8-ff5b-491f-b47b-f507f0e1fcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:26:30.945856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:26:30.985467 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 700/1405] 10.222.0.67 () {58 vars in 1039 bytes} [Thu Jun 30 16:26:30 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:26:30.995810 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba2152-109f-432d-ad51-aa8e4ad4a83c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:26:31.004325 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:26:31.012313 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba2152-109f-432d-ad51-aa8e4ad4a83c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:26:31.017891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc8e20-6519-41d1-858c-882c8f435347 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988b917eea4b48ffae32e23b8555a5c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:26:31.018566 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc8e20-6519-41d1-858c-882c8f435347 service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:31.018855 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc8e20-6519-41d1-858c-882c8f435347 service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:31.019144 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc8e20-6519-41d1-858c-882c8f435347 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:31.028966 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc8e20-6519-41d1-858c-882c8f435347 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:26:31.030583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc8e20-6519-41d1-858c-882c8f435347 service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:26:31.040501 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 706/1406] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 30 16:26:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 30 16:26:31.955407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9bf45-95d6-4da4-910f-e58ad02f2ff1 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:31.956042 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9bf45-95d6-4da4-910f-e58ad02f2ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:31.956042 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9bf45-95d6-4da4-910f-e58ad02f2ff1 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:31.957544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 701/1407] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:26:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:26:31.962769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b5b65-f662-4759-9b56-733ec00999e5 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:31.963256 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b5b65-f662-4759-9b56-733ec00999e5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:31.963365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b5b65-f662-4759-9b56-733ec00999e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:31.982449 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-181b5b65-f662-4759-9b56-733ec00999e5 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:26:31.983690 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-181b5b65-f662-4759-9b56-733ec00999e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:26:31.992159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 707/1408] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:26:32.038677 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9bf45-95d6-4da4-910f-e58ad02f2ff1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:26:32.054075 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9bf45-95d6-4da4-910f-e58ad02f2ff1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:26:32.060091 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a79a68-1e6b-4a0a-9aed-6e09bed26022 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:26:32.060738 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a79a68-1e6b-4a0a-9aed-6e09bed26022 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:32.061024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a79a68-1e6b-4a0a-9aed-6e09bed26022 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:32.061300 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a79a68-1e6b-4a0a-9aed-6e09bed26022 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:32.072748 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a79a68-1e6b-4a0a-9aed-6e09bed26022 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:26:32.074357 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a79a68-1e6b-4a0a-9aed-6e09bed26022 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:26:32.085847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 702/1409] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:26:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:26:41.977545 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=8 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:26:42.081001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=21 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:26:55.429089 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72df77-a967-4827-84df-006ca7cf64f2 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:55.429674 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72df77-a967-4827-84df-006ca7cf64f2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:55.429674 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72df77-a967-4827-84df-006ca7cf64f2 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:55.431010 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 708/1410] 10.222.0.67 () {54 vars in 918 bytes} [Thu Jun 30 16:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:26:55.436442 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a015ff4-349b-4cd5-bc72-99ad96550663 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:55.436848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a015ff4-349b-4cd5-bc72-99ad96550663 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:55.436994 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a015ff4-349b-4cd5-bc72-99ad96550663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:55.455540 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9a015ff4-349b-4cd5-bc72-99ad96550663 None None] MFA Rules not processed for user `a932a9390fdd47f3adcf053a60250944`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:26:55.456762 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9a015ff4-349b-4cd5-bc72-99ad96550663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:26:55.465276 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 703/1411] 10.222.0.67 () {58 vars in 1015 bytes} [Thu Jun 30 16:26:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:26:55.479918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeaf417-c845-4898-8ee5-575ff7a09f1f None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:55.480226 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeaf417-c845-4898-8ee5-575ff7a09f1f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:55.480502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeaf417-c845-4898-8ee5-575ff7a09f1f None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:55.481842 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 709/1412] 10.222.0.67 () {54 vars in 918 bytes} [Thu Jun 30 16:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:26:55.487084 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1b5b0-b7a7-49bb-b146-60e3e5959118 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:55.487367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1b5b0-b7a7-49bb-b146-60e3e5959118 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:55.487689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1b5b0-b7a7-49bb-b146-60e3e5959118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:55.506278 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-80d1b5b0-b7a7-49bb-b146-60e3e5959118 None None] MFA Rules not processed for user `71cca3708386459197abf6c67505bc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:26:55.507500 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-80d1b5b0-b7a7-49bb-b146-60e3e5959118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:26:55.517203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 704/1413] 10.222.0.67 () {58 vars in 1015 bytes} [Thu Jun 30 16:26:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:26:58.097225 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f2499-864f-4759-b985-5264fb6888bb None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:58.097817 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f2499-864f-4759-b985-5264fb6888bb None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:58.097817 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f2499-864f-4759-b985-5264fb6888bb None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:58.099125 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 16:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:26:58.104456 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2a49c-1e16-4d68-b5d4-d0f073136ef0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:58.104951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2a49c-1e16-4d68-b5d4-d0f073136ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:58.105049 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c2a49c-1e16-4d68-b5d4-d0f073136ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:58.123703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-42c2a49c-1e16-4d68-b5d4-d0f073136ef0 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:26:58.125105 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-42c2a49c-1e16-4d68-b5d4-d0f073136ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:26:58.133037 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 16:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:26:58.177221 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f2499-864f-4759-b985-5264fb6888bb None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:26:58.191438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f2499-864f-4759-b985-5264fb6888bb None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:26:58.197324 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a66176-f68f-4a68-ae85-94677f007ffc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:26:58.197971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a66176-f68f-4a68-ae85-94677f007ffc service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:58.198259 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a66176-f68f-4a68-ae85-94677f007ffc service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:58.198547 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a66176-f68f-4a68-ae85-94677f007ffc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:58.207606 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a66176-f68f-4a68-ae85-94677f007ffc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:26:58.209233 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a66176-f68f-4a68-ae85-94677f007ffc service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:26:58.219988 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 711/1416] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:26:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:26:59.149419 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff160125-2af2-4608-a2d9-11898267d392 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:59.149995 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff160125-2af2-4608-a2d9-11898267d392 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:59.149995 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff160125-2af2-4608-a2d9-11898267d392 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:59.151395 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 16:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:26:59.157084 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e192b3-5454-4f78-82c9-c0895d3e9cf3 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:59.157527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e192b3-5454-4f78-82c9-c0895d3e9cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:59.157627 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e192b3-5454-4f78-82c9-c0895d3e9cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:59.177033 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-27e192b3-5454-4f78-82c9-c0895d3e9cf3 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:26:59.178250 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-27e192b3-5454-4f78-82c9-c0895d3e9cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:26:59.186211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 16:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:26:59.340177 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff160125-2af2-4608-a2d9-11898267d392 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:26:59.355365 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff160125-2af2-4608-a2d9-11898267d392 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:26:59.361410 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95fb387-5778-4f56-b9eb-d71b830578a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:26:59.362053 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fb387-5778-4f56-b9eb-d71b830578a1 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:26:59.362333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fb387-5778-4f56-b9eb-d71b830578a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:26:59.362605 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fb387-5778-4f56-b9eb-d71b830578a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:26:59.371157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95fb387-5778-4f56-b9eb-d71b830578a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:26:59.372771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95fb387-5778-4f56-b9eb-d71b830578a1 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:26:59.383191 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 707/1419] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:26:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:27:00.284352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1e880-7493-42d7-9e92-643fc7e8c691 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:00.284894 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1e880-7493-42d7-9e92-643fc7e8c691 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:00.284894 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1e880-7493-42d7-9e92-643fc7e8c691 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:00.286183 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 16:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:27:00.291256 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a643fe4-138d-4144-a862-555c38783e05 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:00.291722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a643fe4-138d-4144-a862-555c38783e05 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:00.291834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a643fe4-138d-4144-a862-555c38783e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:00.310405 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5a643fe4-138d-4144-a862-555c38783e05 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:27:00.311660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5a643fe4-138d-4144-a862-555c38783e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:27:00.320139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 16:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:27:00.474416 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1e880-7493-42d7-9e92-643fc7e8c691 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:27:00.488964 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1e880-7493-42d7-9e92-643fc7e8c691 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:27:00.494314 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0df6e5-bf6f-41cf-bdec-d7690cd3a49e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:27:00.494964 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0df6e5-bf6f-41cf-bdec-d7690cd3a49e service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:00.495250 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0df6e5-bf6f-41cf-bdec-d7690cd3a49e service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:00.495567 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0df6e5-bf6f-41cf-bdec-d7690cd3a49e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:00.505261 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0df6e5-bf6f-41cf-bdec-d7690cd3a49e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:27:00.506892 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0df6e5-bf6f-41cf-bdec-d7690cd3a49e service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:27:00.517773 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 714/1422] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:27:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:27:01.419289 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75806957-84b7-4532-b63b-d32bd7887f7d None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:01.419880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75806957-84b7-4532-b63b-d32bd7887f7d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:01.419880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75806957-84b7-4532-b63b-d32bd7887f7d None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:01.421261 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 16:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:27:01.426327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa330724-4f50-4110-9007-c526b9bb527b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:01.426774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa330724-4f50-4110-9007-c526b9bb527b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:01.426887 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa330724-4f50-4110-9007-c526b9bb527b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:01.445123 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fa330724-4f50-4110-9007-c526b9bb527b None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:27:01.446328 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fa330724-4f50-4110-9007-c526b9bb527b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:27:01.454869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 16:27:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:27:01.605618 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75806957-84b7-4532-b63b-d32bd7887f7d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:27:01.620566 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75806957-84b7-4532-b63b-d32bd7887f7d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:27:01.626898 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581cedcb-1699-42e1-9ab0-6bb80e18ab05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:27:01.627577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cedcb-1699-42e1-9ab0-6bb80e18ab05 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:01.633554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cedcb-1699-42e1-9ab0-6bb80e18ab05 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:01.633554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cedcb-1699-42e1-9ab0-6bb80e18ab05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:01.636770 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581cedcb-1699-42e1-9ab0-6bb80e18ab05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:27:01.638357 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581cedcb-1699-42e1-9ab0-6bb80e18ab05 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:27:01.648619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 710/1425] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:27:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:27:02.518604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622f73a-6400-4042-8899-d84cb50dea75 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:02.519194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622f73a-6400-4042-8899-d84cb50dea75 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:02.519194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622f73a-6400-4042-8899-d84cb50dea75 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:02.520614 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 16:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:27:02.525744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be890d-32b4-4e89-b6ee-7c78a2944c64 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:02.526205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be890d-32b4-4e89-b6ee-7c78a2944c64 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:02.526301 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be890d-32b4-4e89-b6ee-7c78a2944c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:02.544838 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-06be890d-32b4-4e89-b6ee-7c78a2944c64 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:27:02.546039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-06be890d-32b4-4e89-b6ee-7c78a2944c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:27:02.554471 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 16:27:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:27:02.707151 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622f73a-6400-4042-8899-d84cb50dea75 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:27:02.721509 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622f73a-6400-4042-8899-d84cb50dea75 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:27:02.726784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3eada-19b5-46a3-b072-59b5d5b80635 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:27:02.727421 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3eada-19b5-46a3-b072-59b5d5b80635 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:02.727755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3eada-19b5-46a3-b072-59b5d5b80635 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:02.728037 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3eada-19b5-46a3-b072-59b5d5b80635 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:02.736372 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3eada-19b5-46a3-b072-59b5d5b80635 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:27:02.737987 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3eada-19b5-46a3-b072-59b5d5b80635 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:27:02.747452 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 717/1428] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:27:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:27:03.622952 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e46d4-0ed5-4ffd-b2c2-4560ef95461e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:03.623619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e46d4-0ed5-4ffd-b2c2-4560ef95461e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:03.623619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e46d4-0ed5-4ffd-b2c2-4560ef95461e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:03.627881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 16:27:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:27:03.633142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83217ee7-53f4-4e55-bb7d-3a3df1596566 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:03.633578 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83217ee7-53f4-4e55-bb7d-3a3df1596566 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:03.633764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83217ee7-53f4-4e55-bb7d-3a3df1596566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:03.651989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-83217ee7-53f4-4e55-bb7d-3a3df1596566 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:27:03.653488 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-83217ee7-53f4-4e55-bb7d-3a3df1596566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:27:03.662170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 16:27:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:27:03.817720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e46d4-0ed5-4ffd-b2c2-4560ef95461e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:27:03.832675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e46d4-0ed5-4ffd-b2c2-4560ef95461e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:27:03.838035 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be11bc-b6c0-4af9-96a5-ad7d524a1b6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:27:03.838676 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be11bc-b6c0-4af9-96a5-ad7d524a1b6c service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:03.838962 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be11bc-b6c0-4af9-96a5-ad7d524a1b6c service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:03.839238 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be11bc-b6c0-4af9-96a5-ad7d524a1b6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:03.848298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be11bc-b6c0-4af9-96a5-ad7d524a1b6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:27:03.849887 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be11bc-b6c0-4af9-96a5-ad7d524a1b6c service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:27:03.859671 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 713/1431] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:27:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:27:04.737731 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26e454-122a-4e1c-8d0b-3f058b8e2c63 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:04.738264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26e454-122a-4e1c-8d0b-3f058b8e2c63 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:04.738381 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26e454-122a-4e1c-8d0b-3f058b8e2c63 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:04.739721 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 16:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:27:04.744802 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11537706-a58a-433a-af8c-6564d1db164d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:04.745257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11537706-a58a-433a-af8c-6564d1db164d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:04.745356 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11537706-a58a-433a-af8c-6564d1db164d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:04.763833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-11537706-a58a-433a-af8c-6564d1db164d None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:27:04.765742 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-11537706-a58a-433a-af8c-6564d1db164d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:27:04.773829 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 16:27:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:27:04.925619 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26e454-122a-4e1c-8d0b-3f058b8e2c63 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:27:04.940351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26e454-122a-4e1c-8d0b-3f058b8e2c63 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:27:04.946092 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fefd137-36db-4232-a5b8-cd9402d1d650 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:27:04.946747 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fefd137-36db-4232-a5b8-cd9402d1d650 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:04.947028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fefd137-36db-4232-a5b8-cd9402d1d650 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:04.947309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fefd137-36db-4232-a5b8-cd9402d1d650 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:04.956888 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fefd137-36db-4232-a5b8-cd9402d1d650 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:27:04.958512 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fefd137-36db-4232-a5b8-cd9402d1d650 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:27:04.968199 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 720/1434] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:27:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:27:05.866967 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb6c84-fbd4-4ecc-8c1b-cbc4bfe929a9 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:05.867555 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb6c84-fbd4-4ecc-8c1b-cbc4bfe929a9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:05.867555 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb6c84-fbd4-4ecc-8c1b-cbc4bfe929a9 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:05.868964 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 16:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:27:05.874062 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd9ab1-3297-4720-a305-030c4e2fb00c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:05.874516 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd9ab1-3297-4720-a305-030c4e2fb00c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:05.874654 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd9ab1-3297-4720-a305-030c4e2fb00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:05.893104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6bcd9ab1-3297-4720-a305-030c4e2fb00c None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:27:05.894437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6bcd9ab1-3297-4720-a305-030c4e2fb00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:27:05.902243 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 16:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:27:06.056671 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb6c84-fbd4-4ecc-8c1b-cbc4bfe929a9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:27:06.071773 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb6c84-fbd4-4ecc-8c1b-cbc4bfe929a9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:27:06.077001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0cb1b-5ad8-45a1-addd-1fc24ca85368 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:27:06.078638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0cb1b-5ad8-45a1-addd-1fc24ca85368 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:06.078914 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0cb1b-5ad8-45a1-addd-1fc24ca85368 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:06.079187 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0cb1b-5ad8-45a1-addd-1fc24ca85368 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:06.087586 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0cb1b-5ad8-45a1-addd-1fc24ca85368 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:27:06.089161 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0cb1b-5ad8-45a1-addd-1fc24ca85368 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:27:06.099556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 716/1437] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:27:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:27:07.022734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c6097-591b-41c3-99b3-091d0a52170f None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:07.022734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c6097-591b-41c3-99b3-091d0a52170f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:07.022734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346c6097-591b-41c3-99b3-091d0a52170f None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:07.022734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 16:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:27:07.023422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e0b24-dc4f-4877-93ff-7db409bee733 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:07.023422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e0b24-dc4f-4877-93ff-7db409bee733 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:07.023422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e0b24-dc4f-4877-93ff-7db409bee733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:07.035334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b81e0b24-dc4f-4877-93ff-7db409bee733 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:27:07.036568 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b81e0b24-dc4f-4877-93ff-7db409bee733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:27:07.044783 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 16:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:27:07.197707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c6097-591b-41c3-99b3-091d0a52170f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:27:07.213065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346c6097-591b-41c3-99b3-091d0a52170f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:27:07.218941 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e94a0-82c3-4485-94f6-6277640de3b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:27:07.219615 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e94a0-82c3-4485-94f6-6277640de3b4 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:07.219896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e94a0-82c3-4485-94f6-6277640de3b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:07.220169 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e94a0-82c3-4485-94f6-6277640de3b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:07.228878 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e94a0-82c3-4485-94f6-6277640de3b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:27:07.230482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e94a0-82c3-4485-94f6-6277640de3b4 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:27:07.240715 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 723/1440] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:27:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:27:09.211656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724ed01-4680-440a-a648-f1eb6c3c4705 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:09.212221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724ed01-4680-440a-a648-f1eb6c3c4705 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:09.212221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b724ed01-4680-440a-a648-f1eb6c3c4705 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:09.213588 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 16:27:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:27:09.220176 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e20888-5a26-4fc3-a4b8-cdeb5466531d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:09.220700 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e20888-5a26-4fc3-a4b8-cdeb5466531d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:09.220816 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e20888-5a26-4fc3-a4b8-cdeb5466531d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:09.238938 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-63e20888-5a26-4fc3-a4b8-cdeb5466531d None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:27:09.240304 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-63e20888-5a26-4fc3-a4b8-cdeb5466531d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:27:09.248147 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 16:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:27:09.286553 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724ed01-4680-440a-a648-f1eb6c3c4705 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:27:09.297883 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b724ed01-4680-440a-a648-f1eb6c3c4705 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:27:09.301385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c2f4c-fd8e-406e-a7ec-666df54718bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:27:09.302018 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c2f4c-fd8e-406e-a7ec-666df54718bd service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:09.302299 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c2f4c-fd8e-406e-a7ec-666df54718bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:09.302574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c2f4c-fd8e-406e-a7ec-666df54718bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:09.312265 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c2f4c-fd8e-406e-a7ec-666df54718bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:27:09.313721 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76c2f4c-fd8e-406e-a7ec-666df54718bd service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:27:09.323095 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 719/1443] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:27:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:27:11.538289 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd5383-37ae-4703-ae43-b93b34a29ca9 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:11.538816 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd5383-37ae-4703-ae43-b93b34a29ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:11.538816 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd5383-37ae-4703-ae43-b93b34a29ca9 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:11.540185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 16:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:27:11.545514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76719ab8-1be0-4623-b6e7-3b2e7afc3c8e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:11.545941 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76719ab8-1be0-4623-b6e7-3b2e7afc3c8e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:11.546247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76719ab8-1be0-4623-b6e7-3b2e7afc3c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:11.564072 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-76719ab8-1be0-4623-b6e7-3b2e7afc3c8e None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:27:11.565272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-76719ab8-1be0-4623-b6e7-3b2e7afc3c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:27:11.573103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 16:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:27:11.611880 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd5383-37ae-4703-ae43-b93b34a29ca9 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:27:11.623502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd5383-37ae-4703-ae43-b93b34a29ca9 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:27:11.628299 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5572a-3afa-457e-adf0-d88387caa9b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:27:11.628955 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5572a-3afa-457e-adf0-d88387caa9b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:11.629266 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5572a-3afa-457e-adf0-d88387caa9b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:11.629548 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5572a-3afa-457e-adf0-d88387caa9b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:11.638714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd5572a-3afa-457e-adf0-d88387caa9b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:27:11.640836 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd5572a-3afa-457e-adf0-d88387caa9b8 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:27:11.650487 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 726/1446] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:27:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:27:14.159258 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1faa7-1af9-4b15-8e50-ae1b5443a5d1 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:14.159850 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1faa7-1af9-4b15-8e50-ae1b5443a5d1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:14.159850 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1faa7-1af9-4b15-8e50-ae1b5443a5d1 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:14.161232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 16:27:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:27:14.166526 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cc365-9fc6-4bd7-ac96-8aba6a27dfb4 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:14.167027 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cc365-9fc6-4bd7-ac96-8aba6a27dfb4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:14.167148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6cc365-9fc6-4bd7-ac96-8aba6a27dfb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:14.185703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9e6cc365-9fc6-4bd7-ac96-8aba6a27dfb4 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:27:14.188049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9e6cc365-9fc6-4bd7-ac96-8aba6a27dfb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:27:14.199173 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 16:27:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:27:14.238595 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1faa7-1af9-4b15-8e50-ae1b5443a5d1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:27:14.250040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1faa7-1af9-4b15-8e50-ae1b5443a5d1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:27:14.253526 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c269f68b-3c83-451f-8200-588493282a9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:27:14.254166 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269f68b-3c83-451f-8200-588493282a9c service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:14.254450 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269f68b-3c83-451f-8200-588493282a9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:14.254726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c269f68b-3c83-451f-8200-588493282a9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:14.263623 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c269f68b-3c83-451f-8200-588493282a9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:27:14.265170 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c269f68b-3c83-451f-8200-588493282a9c service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:27:14.274527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 722/1449] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:27:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:27:16.476783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e27847-387b-41b9-a7f4-a07f5afaa159 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:16.477348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e27847-387b-41b9-a7f4-a07f5afaa159 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:16.477348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e27847-387b-41b9-a7f4-a07f5afaa159 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:16.478744 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 16:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:27:16.485029 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00853c9d-bcc9-4d46-9248-52f0ed178c84 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:16.485504 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00853c9d-bcc9-4d46-9248-52f0ed178c84 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:16.485611 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00853c9d-bcc9-4d46-9248-52f0ed178c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:16.504110 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-00853c9d-bcc9-4d46-9248-52f0ed178c84 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:27:16.505295 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-00853c9d-bcc9-4d46-9248-52f0ed178c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:27:16.512933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 16:27:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:27:16.552304 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e27847-387b-41b9-a7f4-a07f5afaa159 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:27:16.563728 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e27847-387b-41b9-a7f4-a07f5afaa159 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:27:16.567349 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f72b566-307e-4cd8-bbde-03c4c73beefa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:27:16.568036 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72b566-307e-4cd8-bbde-03c4c73beefa service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:16.568438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72b566-307e-4cd8-bbde-03c4c73beefa service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:16.568723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f72b566-307e-4cd8-bbde-03c4c73beefa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:16.577954 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f72b566-307e-4cd8-bbde-03c4c73beefa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:27:16.579435 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f72b566-307e-4cd8-bbde-03c4c73beefa service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:27:16.589574 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 729/1452] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:27:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:27:18.697608 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8be62-51ef-415b-97d2-b7b098ccd7a8 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:18.698228 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8be62-51ef-415b-97d2-b7b098ccd7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:18.698228 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8be62-51ef-415b-97d2-b7b098ccd7a8 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:18.699567 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 16:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:27:18.705335 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28e3bd-a822-4c28-9fc7-f513374777b7 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:18.705760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28e3bd-a822-4c28-9fc7-f513374777b7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:18.705902 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28e3bd-a822-4c28-9fc7-f513374777b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:18.723830 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7c28e3bd-a822-4c28-9fc7-f513374777b7 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:27:18.725025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7c28e3bd-a822-4c28-9fc7-f513374777b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:27:18.732811 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 16:27:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:27:18.771348 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8be62-51ef-415b-97d2-b7b098ccd7a8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:27:18.782809 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8be62-51ef-415b-97d2-b7b098ccd7a8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:27:18.786332 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d6d3dc-2107-4b2a-8843-0a6a7e3b3840 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:27:18.786967 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6d3dc-2107-4b2a-8843-0a6a7e3b3840 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:18.787246 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6d3dc-2107-4b2a-8843-0a6a7e3b3840 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:18.787554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d6d3dc-2107-4b2a-8843-0a6a7e3b3840 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:18.796512 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d6d3dc-2107-4b2a-8843-0a6a7e3b3840 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:27:18.797931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d6d3dc-2107-4b2a-8843-0a6a7e3b3840 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:27:18.807282 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 725/1455] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:27:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:27:20.919784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb6ab2-09ed-4ad0-83ea-9a54f5572a82 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:20.920906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb6ab2-09ed-4ad0-83ea-9a54f5572a82 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:20.921300 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb6ab2-09ed-4ad0-83ea-9a54f5572a82 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:20.922689 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 16:27:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:27:20.927988 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de23a36-7457-40f1-a294-d3944bc5c770 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:20.928429 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de23a36-7457-40f1-a294-d3944bc5c770 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:20.928538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de23a36-7457-40f1-a294-d3944bc5c770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:20.946449 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2de23a36-7457-40f1-a294-d3944bc5c770 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:27:20.947675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2de23a36-7457-40f1-a294-d3944bc5c770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:27:20.955365 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 16:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:27:20.993236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb6ab2-09ed-4ad0-83ea-9a54f5572a82 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:27:21.005208 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb6ab2-09ed-4ad0-83ea-9a54f5572a82 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:27:21.009532 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622b28c-b285-4e7d-bdbc-3b1af35f824d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:27:21.010203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622b28c-b285-4e7d-bdbc-3b1af35f824d service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:21.010489 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622b28c-b285-4e7d-bdbc-3b1af35f824d service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:21.010766 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622b28c-b285-4e7d-bdbc-3b1af35f824d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:21.020757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622b28c-b285-4e7d-bdbc-3b1af35f824d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:27:21.022221 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622b28c-b285-4e7d-bdbc-3b1af35f824d service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:27:21.032648 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 732/1458] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:27:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:27:23.167915 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097415e-96cd-4723-90d3-540b1c8f74e0 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:23.168525 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097415e-96cd-4723-90d3-540b1c8f74e0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:23.168525 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097415e-96cd-4723-90d3-540b1c8f74e0 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:23.169898 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 16:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:27:23.176039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821033c-d248-4b2a-901f-0542105d9fcf None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:23.176506 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821033c-d248-4b2a-901f-0542105d9fcf None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:23.176610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c821033c-d248-4b2a-901f-0542105d9fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:23.182808 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=70 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:27:23.195085 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c821033c-d248-4b2a-901f-0542105d9fcf None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:27:23.198008 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c821033c-d248-4b2a-901f-0542105d9fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:27:23.205474 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 16:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:27:23.243964 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097415e-96cd-4723-90d3-540b1c8f74e0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:27:23.249581 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=73 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:27:23.255209 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097415e-96cd-4723-90d3-540b1c8f74e0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:27:23.262482 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650963e7-fa90-4504-8667-2b2e642ddd45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:27:23.262482 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650963e7-fa90-4504-8667-2b2e642ddd45 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:23.262482 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650963e7-fa90-4504-8667-2b2e642ddd45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:23.262482 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650963e7-fa90-4504-8667-2b2e642ddd45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:23.269356 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650963e7-fa90-4504-8667-2b2e642ddd45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:27:23.270786 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650963e7-fa90-4504-8667-2b2e642ddd45 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:27:23.281448 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 728/1461] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:27:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:27:25.397966 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85cc10-ffee-45be-9282-16ebe31aad69 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:25.398578 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85cc10-ffee-45be-9282-16ebe31aad69 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:25.398578 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc85cc10-ffee-45be-9282-16ebe31aad69 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:25.399968 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 16:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:27:25.406519 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a9ce9-cbb6-4d04-b5c9-709eba1ff5fb None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:25.406978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a9ce9-cbb6-4d04-b5c9-709eba1ff5fb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:25.407112 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6a9ce9-cbb6-4d04-b5c9-709eba1ff5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:25.424581 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ce6a9ce9-cbb6-4d04-b5c9-709eba1ff5fb None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:27:25.425815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ce6a9ce9-cbb6-4d04-b5c9-709eba1ff5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:27:25.433735 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 16:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:27:25.471916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85cc10-ffee-45be-9282-16ebe31aad69 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:27:25.483089 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc85cc10-ffee-45be-9282-16ebe31aad69 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:27:25.486764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e1c395-4793-4e45-a3a8-d35b6801139a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:27:25.487420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1c395-4793-4e45-a3a8-d35b6801139a service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:25.487732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1c395-4793-4e45-a3a8-d35b6801139a service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:25.488012 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1c395-4793-4e45-a3a8-d35b6801139a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:25.496883 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e1c395-4793-4e45-a3a8-d35b6801139a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:27:25.498501 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e1c395-4793-4e45-a3a8-d35b6801139a service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:27:25.508006 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 735/1464] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:27:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:27:27.612475 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38868f6c-4925-4375-8d20-7342e7bd2a75 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:27.613064 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38868f6c-4925-4375-8d20-7342e7bd2a75 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:27.613064 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38868f6c-4925-4375-8d20-7342e7bd2a75 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:27.614379 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 16:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:27:27.619685 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52ca26-ed67-4447-b064-531805ede1a8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:27.620123 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52ca26-ed67-4447-b064-531805ede1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:27.620237 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52ca26-ed67-4447-b064-531805ede1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:27.638720 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ff52ca26-ed67-4447-b064-531805ede1a8 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:27:27.640413 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ff52ca26-ed67-4447-b064-531805ede1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:27:27.648122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 16:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:27:27.687228 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38868f6c-4925-4375-8d20-7342e7bd2a75 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:27:27.698061 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38868f6c-4925-4375-8d20-7342e7bd2a75 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:27:27.701965 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d8d27-5a1b-4326-a43c-2e23b35ba27d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:27:27.702617 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d8d27-5a1b-4326-a43c-2e23b35ba27d service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:27.702901 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d8d27-5a1b-4326-a43c-2e23b35ba27d service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:27.703174 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d8d27-5a1b-4326-a43c-2e23b35ba27d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:27.712741 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d8d27-5a1b-4326-a43c-2e23b35ba27d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:27:27.714177 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d8d27-5a1b-4326-a43c-2e23b35ba27d service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:27:27.723898 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 731/1467] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:27:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:27:29.818063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e0ec5-c1ba-43e6-8015-949a2e20082b None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:29.818612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e0ec5-c1ba-43e6-8015-949a2e20082b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:29.818612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e0ec5-c1ba-43e6-8015-949a2e20082b None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:29.820028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 16:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:27:29.825339 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c585b-9c09-4d2e-a4f4-cf9faa84914a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:29.825770 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c585b-9c09-4d2e-a4f4-cf9faa84914a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:29.825893 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911c585b-9c09-4d2e-a4f4-cf9faa84914a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:29.845336 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-911c585b-9c09-4d2e-a4f4-cf9faa84914a None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:27:29.846531 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-911c585b-9c09-4d2e-a4f4-cf9faa84914a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:27:29.854389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 16:27:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:27:29.891462 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e0ec5-c1ba-43e6-8015-949a2e20082b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:27:29.902831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e0ec5-c1ba-43e6-8015-949a2e20082b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:27:29.906568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377574d4-656c-45fe-aa7f-d34a35a776b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:27:29.907204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377574d4-656c-45fe-aa7f-d34a35a776b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:29.907991 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377574d4-656c-45fe-aa7f-d34a35a776b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:29.907991 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377574d4-656c-45fe-aa7f-d34a35a776b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:29.916738 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377574d4-656c-45fe-aa7f-d34a35a776b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:27:29.918170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377574d4-656c-45fe-aa7f-d34a35a776b1 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:27:29.928370 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 738/1470] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:27:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:27:39.839214 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=17 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:27:39.924015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=17 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:27:50.382195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c5f71-33b4-45d9-bf44-d4616cb110ee None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:50.382824 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c5f71-33b4-45d9-bf44-d4616cb110ee None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:50.382824 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c5f71-33b4-45d9-bf44-d4616cb110ee None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:50.384228 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 733/1471] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:27:50.390621 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8b0a9-020d-4738-bb9b-e819f844c3a2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:50.391195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8b0a9-020d-4738-bb9b-e819f844c3a2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:50.391195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8b0a9-020d-4738-bb9b-e819f844c3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:50.410065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d5e8b0a9-020d-4738-bb9b-e819f844c3a2 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:27:50.411291 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d5e8b0a9-020d-4738-bb9b-e819f844c3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:27:50.419805 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 739/1472] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:27:50.471100 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082d63c-977a-4212-a5f2-dc0acd7e8770 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:50.471389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082d63c-977a-4212-a5f2-dc0acd7e8770 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:50.471724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082d63c-977a-4212-a5f2-dc0acd7e8770 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:50.473107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 734/1473] 10.222.0.67 () {54 vars in 962 bytes} [Thu Jun 30 16:27:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:27:50.478218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc60d0-7919-4fca-89d7-b08ac9068b03 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:50.478511 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc60d0-7919-4fca-89d7-b08ac9068b03 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:50.478798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc60d0-7919-4fca-89d7-b08ac9068b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:50.498518 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7cbc60d0-7919-4fca-89d7-b08ac9068b03 None None] MFA Rules not processed for user `ec41c84c9375437590c42621ca5b400a`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:27:50.499789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7cbc60d0-7919-4fca-89d7-b08ac9068b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:27:50.506455 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 740/1474] 10.222.0.67 () {58 vars in 1059 bytes} [Thu Jun 30 16:27:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:27:50.514198 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082d63c-977a-4212-a5f2-dc0acd7e8770 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:27:50.525068 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082d63c-977a-4212-a5f2-dc0acd7e8770 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:27:50.529475 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100f81e-c1f9-40f6-8f48-f1408349d5fc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec41c84c9375437590c42621ca5b400a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:27:50.530120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100f81e-c1f9-40f6-8f48-f1408349d5fc service glance] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:50.530402 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100f81e-c1f9-40f6-8f48-f1408349d5fc service glance] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:50.530677 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100f81e-c1f9-40f6-8f48-f1408349d5fc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:50.540039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4100f81e-c1f9-40f6-8f48-f1408349d5fc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:27:50.541474 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4100f81e-c1f9-40f6-8f48-f1408349d5fc service glance] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:27:50.552179 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 735/1475] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:27:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:27:52.727761 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ab37f-6de3-4283-b7bf-4a4ba6396559 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:52.728463 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ab37f-6de3-4283-b7bf-4a4ba6396559 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:52.728463 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ab37f-6de3-4283-b7bf-4a4ba6396559 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:52.729683 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 741/1476] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:27:52.736714 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9039cd0-bfa0-4519-836c-9ff31c3551be None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:52.737328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9039cd0-bfa0-4519-836c-9ff31c3551be None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:52.737610 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9039cd0-bfa0-4519-836c-9ff31c3551be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:52.758433 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c9039cd0-bfa0-4519-836c-9ff31c3551be None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:27:52.760039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c9039cd0-bfa0-4519-836c-9ff31c3551be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:27:52.769531 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 736/1477] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:27:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:27:52.795568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ab37f-6de3-4283-b7bf-4a4ba6396559 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:27:52.809755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ab37f-6de3-4283-b7bf-4a4ba6396559 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:27:52.814965 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af6ab1-4e9c-456f-830e-380c2b17d5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:27:52.815675 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af6ab1-4e9c-456f-830e-380c2b17d5ee admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:52.815955 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af6ab1-4e9c-456f-830e-380c2b17d5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:52.816243 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af6ab1-4e9c-456f-830e-380c2b17d5ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:52.816977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af6ab1-4e9c-456f-830e-380c2b17d5ee admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:27:52.818801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: WARNING py.warnings [None req-94af6ab1-4e9c-456f-830e-380c2b17d5ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 16:27:52.818801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: warnings.warn(msg) Jun 30 16:27:52.818801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: Jun 30 16:27:52.820019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af6ab1-4e9c-456f-830e-380c2b17d5ee admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:27:52.844021 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 742/1478] 10.222.0.67 () {56 vars in 1108 bytes} [Thu Jun 30 16:27:52 2022] GET /identity/v3/projects => generated 2553 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:27:54.989929 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51693b3-cf3d-4c29-a37c-b154c83d6896 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:54.990490 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51693b3-cf3d-4c29-a37c-b154c83d6896 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:54.990490 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51693b3-cf3d-4c29-a37c-b154c83d6896 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:54.991902 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 737/1479] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:27:54.997677 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805794a-c9d4-4c3a-a32c-330edcde5de8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:54.998198 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805794a-c9d4-4c3a-a32c-330edcde5de8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:55.001298 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a805794a-c9d4-4c3a-a32c-330edcde5de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:55.019870 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a805794a-c9d4-4c3a-a32c-330edcde5de8 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:27:55.021100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a805794a-c9d4-4c3a-a32c-330edcde5de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:27:55.030440 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 743/1480] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:27:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:27:55.075182 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51693b3-cf3d-4c29-a37c-b154c83d6896 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:27:55.089492 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51693b3-cf3d-4c29-a37c-b154c83d6896 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:27:55.095830 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604b8a8-2256-4fcc-abf8-b634b9e64625 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:27:55.096687 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604b8a8-2256-4fcc-abf8-b634b9e64625 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:55.096855 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604b8a8-2256-4fcc-abf8-b634b9e64625 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:55.097026 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604b8a8-2256-4fcc-abf8-b634b9e64625 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:55.106737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604b8a8-2256-4fcc-abf8-b634b9e64625 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:27:55.108470 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604b8a8-2256-4fcc-abf8-b634b9e64625 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:27:55.119306 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 738/1481] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:27:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:27:57.431306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707129b-c79c-4a09-9219-932ed5ce216a None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:57.431972 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707129b-c79c-4a09-9219-932ed5ce216a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:57.431972 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707129b-c79c-4a09-9219-932ed5ce216a None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:57.433250 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 744/1482] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:27:57.439027 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee407f5-4d75-4f3d-b7fe-d13b87509352 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:57.439619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee407f5-4d75-4f3d-b7fe-d13b87509352 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:57.439619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee407f5-4d75-4f3d-b7fe-d13b87509352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:57.457507 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5ee407f5-4d75-4f3d-b7fe-d13b87509352 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:27:57.458733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5ee407f5-4d75-4f3d-b7fe-d13b87509352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:27:57.467113 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 739/1483] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:27:57.504276 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707129b-c79c-4a09-9219-932ed5ce216a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:27:57.515568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707129b-c79c-4a09-9219-932ed5ce216a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:27:57.519081 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b99772-236b-4649-9a4f-f0da3bff0884 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:27:57.519770 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b99772-236b-4649-9a4f-f0da3bff0884 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:57.520053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b99772-236b-4649-9a4f-f0da3bff0884 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:57.520333 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b99772-236b-4649-9a4f-f0da3bff0884 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:57.530980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b99772-236b-4649-9a4f-f0da3bff0884 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:27:57.532452 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b99772-236b-4649-9a4f-f0da3bff0884 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:27:57.542109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 745/1484] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:27:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:27:59.720881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2511d36-1351-4900-b3b8-8750bf38224e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:59.721455 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2511d36-1351-4900-b3b8-8750bf38224e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:59.721455 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2511d36-1351-4900-b3b8-8750bf38224e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:59.722846 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 740/1485] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:27:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:27:59.728131 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466e4e0-bfbd-4ad6-afc8-ab7270cb2951 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:59.728549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466e4e0-bfbd-4ad6-afc8-ab7270cb2951 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:59.728690 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466e4e0-bfbd-4ad6-afc8-ab7270cb2951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:59.747374 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e466e4e0-bfbd-4ad6-afc8-ab7270cb2951 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:27:59.748625 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e466e4e0-bfbd-4ad6-afc8-ab7270cb2951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:27:59.756527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 746/1486] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:27:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:27:59.796823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2511d36-1351-4900-b3b8-8750bf38224e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:27:59.807849 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2511d36-1351-4900-b3b8-8750bf38224e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:27:59.811399 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea2dec-a605-4f56-8a26-950dade566f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:27:59.812082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea2dec-a605-4f56-8a26-950dade566f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:27:59.812391 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea2dec-a605-4f56-8a26-950dade566f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:27:59.812670 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea2dec-a605-4f56-8a26-950dade566f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:27:59.822151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea2dec-a605-4f56-8a26-950dade566f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:27:59.823625 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea2dec-a605-4f56-8a26-950dade566f4 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:27:59.833696 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 741/1487] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:27:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:28:02.082623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a29b46-47be-4755-904f-736acd1bc4b8 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:28:02.083183 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a29b46-47be-4755-904f-736acd1bc4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:28:02.083183 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a29b46-47be-4755-904f-736acd1bc4b8 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:28:02.084593 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 747/1488] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:28:02.090793 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6ac47-e2ab-47e1-b4ef-572c0806f9cb None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:28:02.091249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6ac47-e2ab-47e1-b4ef-572c0806f9cb None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:28:02.091360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a6ac47-e2ab-47e1-b4ef-572c0806f9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:28:02.109529 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-22a6ac47-e2ab-47e1-b4ef-572c0806f9cb None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:28:02.110740 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-22a6ac47-e2ab-47e1-b4ef-572c0806f9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:28:02.120570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 742/1489] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:28:02.166578 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a29b46-47be-4755-904f-736acd1bc4b8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:28:02.178364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a29b46-47be-4755-904f-736acd1bc4b8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:28:02.181835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c74e91-57f5-4139-8611-225f0db25cce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:28:02.182505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c74e91-57f5-4139-8611-225f0db25cce service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:28:02.182792 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c74e91-57f5-4139-8611-225f0db25cce service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:28:02.183067 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c74e91-57f5-4139-8611-225f0db25cce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:28:02.193369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c74e91-57f5-4139-8611-225f0db25cce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:28:02.194831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c74e91-57f5-4139-8611-225f0db25cce service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:28:02.205681 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 748/1490] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:28:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:28:04.620943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d33d1a-596e-4384-9340-57478cebc635 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:28:04.621630 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d33d1a-596e-4384-9340-57478cebc635 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:28:04.621630 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d33d1a-596e-4384-9340-57478cebc635 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:28:04.623069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 743/1491] 10.222.0.67 () {54 vars in 926 bytes} [Thu Jun 30 16:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:28:04.628436 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d0c85-5b7e-4ea7-9c5f-bc90ee909fd9 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:28:04.630294 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d0c85-5b7e-4ea7-9c5f-bc90ee909fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:28:04.630294 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d0c85-5b7e-4ea7-9c5f-bc90ee909fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:28:04.648068 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-aa2d0c85-5b7e-4ea7-9c5f-bc90ee909fd9 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:28:04.649358 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-aa2d0c85-5b7e-4ea7-9c5f-bc90ee909fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:28:04.658333 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 749/1492] 10.222.0.67 () {58 vars in 1023 bytes} [Thu Jun 30 16:28:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:28:04.701509 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d33d1a-596e-4384-9340-57478cebc635 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:28:04.712787 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d33d1a-596e-4384-9340-57478cebc635 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:28:04.717406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53be548-fcf7-4284-b632-311e87b0f74e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:28:04.718051 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53be548-fcf7-4284-b632-311e87b0f74e service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:28:04.718335 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53be548-fcf7-4284-b632-311e87b0f74e service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:28:04.718615 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53be548-fcf7-4284-b632-311e87b0f74e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:28:04.727986 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53be548-fcf7-4284-b632-311e87b0f74e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:28:04.729477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53be548-fcf7-4284-b632-311e87b0f74e service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:28:04.739417 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 744/1493] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:28:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:28:14.642760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=33 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:28:14.735235 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=33 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:29:53.259768 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d0f32-345b-48bb-b0a1-bd91880da7e0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:29:53.260479 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d0f32-345b-48bb-b0a1-bd91880da7e0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:29:53.260479 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d0f32-345b-48bb-b0a1-bd91880da7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:29:53.280219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-dc1d0f32-345b-48bb-b0a1-bd91880da7e0 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:29:53.281440 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-dc1d0f32-345b-48bb-b0a1-bd91880da7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:29:53.289737 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 750/1494] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:29:53.298393 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bdc25-1885-4290-a65a-ba13e59b7d72 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:29:53.299038 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bdc25-1885-4290-a65a-ba13e59b7d72 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:29:53.299038 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bdc25-1885-4290-a65a-ba13e59b7d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:29:53.319913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c81bdc25-1885-4290-a65a-ba13e59b7d72 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:29:53.321174 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c81bdc25-1885-4290-a65a-ba13e59b7d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:29:53.329361 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 745/1495] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:29:53.338218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d0f32-345b-48bb-b0a1-bd91880da7e0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:29:53.358200 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d0f32-345b-48bb-b0a1-bd91880da7e0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:29:53.363229 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f537b-73dd-4b1a-b8ee-c1976987c630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:29:53.363933 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f537b-73dd-4b1a-b8ee-c1976987c630 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:29:53.364255 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f537b-73dd-4b1a-b8ee-c1976987c630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:29:53.364529 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2f537b-73dd-4b1a-b8ee-c1976987c630 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:29:53.365367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f537b-73dd-4b1a-b8ee-c1976987c630 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:29:53.365668 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f537b-73dd-4b1a-b8ee-c1976987c630 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:29:53.366717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: WARNING py.warnings [None req-6c2f537b-73dd-4b1a-b8ee-c1976987c630 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 16:29:53.366717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: warnings.warn(msg) Jun 30 16:29:53.366717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: Jun 30 16:29:53.367641 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2f537b-73dd-4b1a-b8ee-c1976987c630 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:29:53.380857 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 751/1496] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:29:53.389712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81bdc25-1885-4290-a65a-ba13e59b7d72 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:29:53.400324 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81bdc25-1885-4290-a65a-ba13e59b7d72 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:29:53.405023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204c0be-b72c-43bd-ab0e-2d191104e424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:29:53.405744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204c0be-b72c-43bd-ab0e-2d191104e424 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:29:53.406023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204c0be-b72c-43bd-ab0e-2d191104e424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:29:53.406298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204c0be-b72c-43bd-ab0e-2d191104e424 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:29:53.406996 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204c0be-b72c-43bd-ab0e-2d191104e424 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:29:53.408944 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-c204c0be-b72c-43bd-ab0e-2d191104e424 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 16:29:53.408944 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Jun 30 16:29:53.408944 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: Jun 30 16:29:53.410169 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204c0be-b72c-43bd-ab0e-2d191104e424 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:29:53.546934 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 746/1497] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:29:53 2022] POST /identity/v3/projects => generated 379 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:29:53.551406 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f537b-73dd-4b1a-b8ee-c1976987c630 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:29:53.563119 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2f537b-73dd-4b1a-b8ee-c1976987c630 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:29:53.634561 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2fb45f-2716-4903-ba27-4814e3b7459a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:29:53.635245 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fb45f-2716-4903-ba27-4814e3b7459a admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:29:53.635572 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fb45f-2716-4903-ba27-4814e3b7459a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:29:53.635856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2fb45f-2716-4903-ba27-4814e3b7459a admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:29:53.636581 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2fb45f-2716-4903-ba27-4814e3b7459a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:29:53.637633 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: WARNING py.warnings [None req-0a2fb45f-2716-4903-ba27-4814e3b7459a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 16:29:53.637633 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: warnings.warn(msg) Jun 30 16:29:53.637633 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: Jun 30 16:29:53.638869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2fb45f-2716-4903-ba27-4814e3b7459a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:29:53.746437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 752/1498] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:29:53 2022] POST /identity/v3/users => generated 290 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:29:53.750689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204c0be-b72c-43bd-ab0e-2d191104e424 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:29:53.762613 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204c0be-b72c-43bd-ab0e-2d191104e424 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:29:53.767525 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc1f0e-522d-47a8-9b1b-0cc19d7d3762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:29:53.768226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc1f0e-522d-47a8-9b1b-0cc19d7d3762 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:29:53.768508 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc1f0e-522d-47a8-9b1b-0cc19d7d3762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:29:53.768942 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc1f0e-522d-47a8-9b1b-0cc19d7d3762 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:29:53.769687 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc1f0e-522d-47a8-9b1b-0cc19d7d3762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:29:53.770738 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-61dc1f0e-522d-47a8-9b1b-0cc19d7d3762 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 16:29:53.770738 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Jun 30 16:29:53.770738 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: Jun 30 16:29:53.771668 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc1f0e-522d-47a8-9b1b-0cc19d7d3762 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:29:53.786704 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 747/1499] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:29:53.790962 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2fb45f-2716-4903-ba27-4814e3b7459a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:29:53.802995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2fb45f-2716-4903-ba27-4814e3b7459a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:29:53.807978 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a0b529-e0b1-4333-a116-5530d84dc9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:29:53.808677 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0b529-e0b1-4333-a116-5530d84dc9ee admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:29:53.808962 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0b529-e0b1-4333-a116-5530d84dc9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:29:53.809237 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a0b529-e0b1-4333-a116-5530d84dc9ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:29:53.809983 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0b529-e0b1-4333-a116-5530d84dc9ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:29:53.811035 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: WARNING py.warnings [None req-60a0b529-e0b1-4333-a116-5530d84dc9ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 16:29:53.811035 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: warnings.warn(msg) Jun 30 16:29:53.811035 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: Jun 30 16:29:53.812053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a0b529-e0b1-4333-a116-5530d84dc9ee admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:29:53.827591 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 753/1500] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:29:53.832301 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc1f0e-522d-47a8-9b1b-0cc19d7d3762 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:29:53.843203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc1f0e-522d-47a8-9b1b-0cc19d7d3762 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:29:53.848186 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c33111-36d6-432b-b4a0-e56be3b77f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:29:53.849689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c33111-36d6-432b-b4a0-e56be3b77f69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:29:53.849998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c33111-36d6-432b-b4a0-e56be3b77f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:29:53.850247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c33111-36d6-432b-b4a0-e56be3b77f69 admin admin] PATH_INFO: `/v3/projects/392ae952a26f47c48914edeff1528a7a/users/296ba9e548094006858148e748b84fa0/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:29:53.868380 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c33111-36d6-432b-b4a0-e56be3b77f69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=392ae952a26f47c48914edeff1528a7a, user_id=296ba9e548094006858148e748b84fa0, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:29:53.869629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-e1c33111-36d6-432b-b4a0-e56be3b77f69 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 16:29:53.869629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Jun 30 16:29:53.869629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: Jun 30 16:29:53.871782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c33111-36d6-432b-b4a0-e56be3b77f69 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:29:53.968909 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 748/1501] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:29:53 2022] PUT /identity/v3/projects/392ae952a26f47c48914edeff1528a7a/users/296ba9e548094006858148e748b84fa0/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:29:53.983061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e77994-aacf-4973-9b30-4c86a0d92676 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:29:53.983360 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e77994-aacf-4973-9b30-4c86a0d92676 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:29:53.983679 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e77994-aacf-4973-9b30-4c86a0d92676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:29:54.021808 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-25e77994-aacf-4973-9b30-4c86a0d92676 None None] MFA Rules not processed for user `296ba9e548094006858148e748b84fa0`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:29:54.023106 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-25e77994-aacf-4973-9b30-4c86a0d92676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:29:54.109530 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 754/1502] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:29:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:29:54.124975 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035a7d3-c4bf-485c-a28d-7e569bf21963 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:29:54.125279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035a7d3-c4bf-485c-a28d-7e569bf21963 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:29:54.125571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035a7d3-c4bf-485c-a28d-7e569bf21963 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:29:54.127121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 749/1503] 10.222.0.67 () {54 vars in 963 bytes} [Thu Jun 30 16:29:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:29:54.133971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07107a-ca13-4bb2-9593-ed9d78535a8d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:29:54.134489 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07107a-ca13-4bb2-9593-ed9d78535a8d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:29:54.134993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07107a-ca13-4bb2-9593-ed9d78535a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:29:54.167191 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-db07107a-ca13-4bb2-9593-ed9d78535a8d None None] MFA Rules not processed for user `61437a2fa2a94b35b1b4a5750f6e2cad`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:29:54.169447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-db07107a-ca13-4bb2-9593-ed9d78535a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:29:54.237229 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:29:54.245852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 755/1504] 10.222.0.67 () {58 vars in 1060 bytes} [Thu Jun 30 16:29:54 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:29:54.253887 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035a7d3-c4bf-485c-a28d-7e569bf21963 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:29:54.266649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035a7d3-c4bf-485c-a28d-7e569bf21963 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:29:54.277063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0347c25-f485-4692-8531-b40c2fe8cae4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:29:54.277063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0347c25-f485-4692-8531-b40c2fe8cae4 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:29:54.277063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0347c25-f485-4692-8531-b40c2fe8cae4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:29:54.277063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0347c25-f485-4692-8531-b40c2fe8cae4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:29:54.282899 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0347c25-f485-4692-8531-b40c2fe8cae4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:29:54.285308 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0347c25-f485-4692-8531-b40c2fe8cae4 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:29:54.302474 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 750/1505] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:29:54 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:29:54.347554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a322c-a97d-47e9-a9fa-c3822aee0e1f None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:29:54.348089 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a322c-a97d-47e9-a9fa-c3822aee0e1f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:29:54.348318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a322c-a97d-47e9-a9fa-c3822aee0e1f None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:29:54.350450 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 756/1506] 10.222.0.67 () {52 vars in 829 bytes} [Thu Jun 30 16:29:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:29:54.354565 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0347c25-f485-4692-8531-b40c2fe8cae4 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:29:54.421136 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0347c25-f485-4692-8531-b40c2fe8cae4 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:29:54.425820 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682ef10-9ba2-44e8-8d20-84721adf2623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:29:54.427223 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682ef10-9ba2-44e8-8d20-84721adf2623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:29:54.427542 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682ef10-9ba2-44e8-8d20-84721adf2623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:29:54.427823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a682ef10-9ba2-44e8-8d20-84721adf2623 admin admin] PATH_INFO: `/v3/users/296ba9e548094006858148e748b84fa0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:29:54.429107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682ef10-9ba2-44e8-8d20-84721adf2623 admin admin] RBAC: Authorizing `identity:delete_user(user_id=296ba9e548094006858148e748b84fa0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:29:54.430125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-a682ef10-9ba2-44e8-8d20-84721adf2623 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 16:29:54.430125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Jun 30 16:29:54.430125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: Jun 30 16:29:54.431354 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a682ef10-9ba2-44e8-8d20-84721adf2623 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:29:54.645483 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a682ef10-9ba2-44e8-8d20-84721adf2623 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '296ba9e548094006858148e748b84fa0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:29:54.647003 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a682ef10-9ba2-44e8-8d20-84721adf2623 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '296ba9e548094006858148e748b84fa0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:29:54.721932 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a682ef10-9ba2-44e8-8d20-84721adf2623 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '296ba9e548094006858148e748b84fa0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:29:54.741764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a682ef10-9ba2-44e8-8d20-84721adf2623 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '296ba9e548094006858148e748b84fa0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:29:54.742969 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a682ef10-9ba2-44e8-8d20-84721adf2623 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '296ba9e548094006858148e748b84fa0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:29:54.775668 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:29:54.790831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 751/1507] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:29:54 2022] DELETE /identity/v3/users/296ba9e548094006858148e748b84fa0 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:29:54.797032 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02267e62-894f-4863-abb2-c4a9cd456292 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:29:54.797381 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02267e62-894f-4863-abb2-c4a9cd456292 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:29:54.797730 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02267e62-894f-4863-abb2-c4a9cd456292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:29:54.820572 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-02267e62-894f-4863-abb2-c4a9cd456292 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:29:54.822267 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-02267e62-894f-4863-abb2-c4a9cd456292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:29:54.895063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 757/1508] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:29:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:29:54.908042 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682ef10-9ba2-44e8-8d20-84721adf2623 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:29:54.909958 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a682ef10-9ba2-44e8-8d20-84721adf2623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:29:55.000634 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a682ef10-9ba2-44e8-8d20-84721adf2623 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:29:55.006707 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6251eb37-970c-4cca-b577-36890ec7386e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:29:55.007597 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251eb37-970c-4cca-b577-36890ec7386e service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:29:55.007980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251eb37-970c-4cca-b577-36890ec7386e service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:29:55.008353 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251eb37-970c-4cca-b577-36890ec7386e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:29:55.021574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251eb37-970c-4cca-b577-36890ec7386e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:29:55.023250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6251eb37-970c-4cca-b577-36890ec7386e service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:29:55.034576 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 752/1509] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:29:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:29:55.660565 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02267e62-894f-4863-abb2-c4a9cd456292 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:29:55.662991 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-02267e62-894f-4863-abb2-c4a9cd456292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:29:55.679674 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02267e62-894f-4863-abb2-c4a9cd456292 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:29:55.688297 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada896a3-daae-4fee-9b4d-cf7ef6ac5349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:29:55.689057 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada896a3-daae-4fee-9b4d-cf7ef6ac5349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:29:55.689355 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada896a3-daae-4fee-9b4d-cf7ef6ac5349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:29:55.689625 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada896a3-daae-4fee-9b4d-cf7ef6ac5349 admin admin] PATH_INFO: `/v3/projects/392ae952a26f47c48914edeff1528a7a` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:29:55.690965 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada896a3-daae-4fee-9b4d-cf7ef6ac5349 admin admin] RBAC: Authorizing `identity:delete_project(project_id=392ae952a26f47c48914edeff1528a7a)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:29:55.692064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: WARNING py.warnings [None req-ada896a3-daae-4fee-9b4d-cf7ef6ac5349 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 16:29:55.692064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: warnings.warn(msg) Jun 30 16:29:55.692064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: Jun 30 16:29:55.693818 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada896a3-daae-4fee-9b4d-cf7ef6ac5349 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:29:55.835635 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ada896a3-daae-4fee-9b4d-cf7ef6ac5349 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '392ae952a26f47c48914edeff1528a7a', 'request_id': 'req-ada896a3-daae-4fee-9b4d-cf7ef6ac5349'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:29:55.961631 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ada896a3-daae-4fee-9b4d-cf7ef6ac5349 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '392ae952a26f47c48914edeff1528a7a', 'request_id': 'req-ada896a3-daae-4fee-9b4d-cf7ef6ac5349'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:29:55.976557 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ada896a3-daae-4fee-9b4d-cf7ef6ac5349 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '392ae952a26f47c48914edeff1528a7a', 'request_id': 'req-ada896a3-daae-4fee-9b4d-cf7ef6ac5349'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:29:55.991357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ada896a3-daae-4fee-9b4d-cf7ef6ac5349 admin admin] The token cache is being invalidate because project 392ae952a26f47c48914edeff1528a7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:29:55.991712 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ada896a3-daae-4fee-9b4d-cf7ef6ac5349 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:29:55.992562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ada896a3-daae-4fee-9b4d-cf7ef6ac5349 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:29:55.995111 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 758/1510] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:29:55 2022] DELETE /identity/v3/projects/392ae952a26f47c48914edeff1528a7a => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:30:05.029782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:30:05.987248 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:31:35.245644 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b6fcc-573b-4bc2-a8bd-eec6a7580d5c None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:31:35.246585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b6fcc-573b-4bc2-a8bd-eec6a7580d5c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:31:35.246585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b6fcc-573b-4bc2-a8bd-eec6a7580d5c None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:31:35.249357 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 16:31:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:31:35.255216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d5d23-e4d4-4e04-a744-2cdef1d95ee4 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:31:35.256122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d5d23-e4d4-4e04-a744-2cdef1d95ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:31:35.256122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d5d23-e4d4-4e04-a744-2cdef1d95ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:31:35.278460 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-828d5d23-e4d4-4e04-a744-2cdef1d95ee4 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:31:35.279740 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-828d5d23-e4d4-4e04-a744-2cdef1d95ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:31:35.355498 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 16:31:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:31:35.516944 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b6fcc-573b-4bc2-a8bd-eec6a7580d5c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:31:35.519024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-883b6fcc-573b-4bc2-a8bd-eec6a7580d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:31:35.609481 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b6fcc-573b-4bc2-a8bd-eec6a7580d5c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:31:35.616279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e11696d-7bc6-4b94-9a43-8973bd401e91 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:31:35.619248 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11696d-7bc6-4b94-9a43-8973bd401e91 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:31:35.619248 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11696d-7bc6-4b94-9a43-8973bd401e91 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:31:35.619248 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e11696d-7bc6-4b94-9a43-8973bd401e91 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:31:35.628526 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e11696d-7bc6-4b94-9a43-8973bd401e91 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:31:35.630642 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e11696d-7bc6-4b94-9a43-8973bd401e91 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:31:35.643577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 754/1513] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:31:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:31:36.625070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21feb1b-ebbe-4f4e-b6aa-f7de6b8af649 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:31:36.625701 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21feb1b-ebbe-4f4e-b6aa-f7de6b8af649 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:31:36.625701 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21feb1b-ebbe-4f4e-b6aa-f7de6b8af649 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:31:36.627679 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 16:31:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:31:36.634474 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae25314-2117-46a4-ae12-a62f465e2fe6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:31:36.635310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae25314-2117-46a4-ae12-a62f465e2fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:31:36.635310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae25314-2117-46a4-ae12-a62f465e2fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:31:36.658811 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4ae25314-2117-46a4-ae12-a62f465e2fe6 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:31:36.660119 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4ae25314-2117-46a4-ae12-a62f465e2fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:31:36.668658 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 16:31:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:31:36.872719 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21feb1b-ebbe-4f4e-b6aa-f7de6b8af649 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:31:36.890426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21feb1b-ebbe-4f4e-b6aa-f7de6b8af649 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:31:36.897003 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ccb498-62d7-4729-9a2f-06f3d83437ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:31:36.897684 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ccb498-62d7-4729-9a2f-06f3d83437ef service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:31:36.897966 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ccb498-62d7-4729-9a2f-06f3d83437ef service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:31:36.898242 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ccb498-62d7-4729-9a2f-06f3d83437ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:31:36.909196 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ccb498-62d7-4729-9a2f-06f3d83437ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:31:36.912529 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ccb498-62d7-4729-9a2f-06f3d83437ef service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:31:36.931308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 761/1516] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:31:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:31:37.015785 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd7a6e-64cc-4168-b369-13796dfd3efa None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:31:37.016277 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd7a6e-64cc-4168-b369-13796dfd3efa None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:31:37.016583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd7a6e-64cc-4168-b369-13796dfd3efa None None] PATH_INFO: `/v3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:31:37.018473 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 756/1517] 10.222.0.65 () {54 vars in 925 bytes} [Thu Jun 30 16:31:37 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:31:37.026136 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77642a0-6fa4-4ae2-a02b-fa0483f80c68 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:31:37.040580 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77642a0-6fa4-4ae2-a02b-fa0483f80c68 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:31:37.040580 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77642a0-6fa4-4ae2-a02b-fa0483f80c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:31:37.051965 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b77642a0-6fa4-4ae2-a02b-fa0483f80c68 None None] MFA Rules not processed for user `a932a9390fdd47f3adcf053a60250944`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:31:37.053370 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b77642a0-6fa4-4ae2-a02b-fa0483f80c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:31:37.142549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 762/1518] 10.222.0.65 () {58 vars in 1015 bytes} [Thu Jun 30 16:31:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:31:41.301188 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c734fc-b6f9-4759-9186-f7e0e8d1f57c None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:31:41.301789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c734fc-b6f9-4759-9186-f7e0e8d1f57c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:31:41.301789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c734fc-b6f9-4759-9186-f7e0e8d1f57c None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:31:41.303666 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 757/1519] 10.222.0.67 () {54 vars in 960 bytes} [Thu Jun 30 16:31:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:31:41.309602 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abad8b4-b1d0-4181-8187-2d22dc2fec28 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:31:41.310193 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abad8b4-b1d0-4181-8187-2d22dc2fec28 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:31:41.310193 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abad8b4-b1d0-4181-8187-2d22dc2fec28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:31:41.330699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6abad8b4-b1d0-4181-8187-2d22dc2fec28 None None] MFA Rules not processed for user `a932a9390fdd47f3adcf053a60250944`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:31:41.332065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6abad8b4-b1d0-4181-8187-2d22dc2fec28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:31:41.341337 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 763/1520] 10.222.0.67 () {58 vars in 1057 bytes} [Thu Jun 30 16:31:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:31:41.356505 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c734fc-b6f9-4759-9186-f7e0e8d1f57c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:31:41.371493 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c734fc-b6f9-4759-9186-f7e0e8d1f57c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:31:41.377048 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e0936-6a94-4fcd-bdd6-4dcd6e3ad0d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a932a9390fdd47f3adcf053a60250944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:31:41.377782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e0936-6a94-4fcd-bdd6-4dcd6e3ad0d0 service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:31:41.378070 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e0936-6a94-4fcd-bdd6-4dcd6e3ad0d0 service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:31:41.378354 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e0936-6a94-4fcd-bdd6-4dcd6e3ad0d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:31:41.387976 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e0936-6a94-4fcd-bdd6-4dcd6e3ad0d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:31:41.389606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e0936-6a94-4fcd-bdd6-4dcd6e3ad0d0 service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:31:41.401678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 758/1521] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:31:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:31:48.278591 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61ad5c-680a-4ded-abc6-7e61acb94ef3 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:31:48.279173 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61ad5c-680a-4ded-abc6-7e61acb94ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:31:48.279173 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d61ad5c-680a-4ded-abc6-7e61acb94ef3 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:31:48.280584 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 764/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 16:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:31:48.286918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336740c-047c-4956-9a3c-805b135bf8ec None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:31:48.287537 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336740c-047c-4956-9a3c-805b135bf8ec None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:31:48.287807 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336740c-047c-4956-9a3c-805b135bf8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:31:48.307426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4336740c-047c-4956-9a3c-805b135bf8ec None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:31:48.308727 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4336740c-047c-4956-9a3c-805b135bf8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:31:48.317792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 759/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 16:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:31:48.476908 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61ad5c-680a-4ded-abc6-7e61acb94ef3 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:31:48.493527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d61ad5c-680a-4ded-abc6-7e61acb94ef3 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:31:48.499536 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a191beda-a39a-498c-9f22-ddcfa3c546c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:31:48.500207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191beda-a39a-498c-9f22-ddcfa3c546c7 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:31:48.500945 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191beda-a39a-498c-9f22-ddcfa3c546c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:31:48.501231 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a191beda-a39a-498c-9f22-ddcfa3c546c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:31:48.511366 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191beda-a39a-498c-9f22-ddcfa3c546c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:31:48.513092 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a191beda-a39a-498c-9f22-ddcfa3c546c7 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:31:48.524437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 765/1524] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:31:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:31:49.417043 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3864e-2163-4300-b018-13bf52deb53a None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:31:49.417675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3864e-2163-4300-b018-13bf52deb53a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:31:49.417872 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3864e-2163-4300-b018-13bf52deb53a None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:31:49.419498 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 760/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 16:31:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:31:49.425380 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b2de5-4fc9-4edb-af51-3cebf180d6ff None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:31:49.426110 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b2de5-4fc9-4edb-af51-3cebf180d6ff None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:31:49.426110 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b2de5-4fc9-4edb-af51-3cebf180d6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:31:49.450546 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-203b2de5-4fc9-4edb-af51-3cebf180d6ff None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:31:49.451849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-203b2de5-4fc9-4edb-af51-3cebf180d6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:31:49.461026 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 766/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 16:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:31:49.621793 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3864e-2163-4300-b018-13bf52deb53a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:31:49.642109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3864e-2163-4300-b018-13bf52deb53a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:31:49.649024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1250d-66cc-4014-ace3-55b185e583e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:31:49.649701 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1250d-66cc-4014-ace3-55b185e583e8 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:31:49.649989 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1250d-66cc-4014-ace3-55b185e583e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:31:49.650277 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1250d-66cc-4014-ace3-55b185e583e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:31:49.661147 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1250d-66cc-4014-ace3-55b185e583e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:31:49.662800 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1250d-66cc-4014-ace3-55b185e583e8 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:31:49.675009 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 761/1527] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:31:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:31:53.950490 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ffd843-b579-48dd-9e5a-85e58ca20503 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:31:53.951057 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ffd843-b579-48dd-9e5a-85e58ca20503 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:31:53.951057 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ffd843-b579-48dd-9e5a-85e58ca20503 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:31:53.952429 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 767/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 16:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:31:53.959448 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8f560-9b04-46fd-a492-04a07c939f21 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:31:53.960295 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8f560-9b04-46fd-a492-04a07c939f21 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:31:53.960493 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8f560-9b04-46fd-a492-04a07c939f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:31:53.980961 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fcb8f560-9b04-46fd-a492-04a07c939f21 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:31:53.982195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fcb8f560-9b04-46fd-a492-04a07c939f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:31:53.991088 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 762/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 16:31:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:31:54.036438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ffd843-b579-48dd-9e5a-85e58ca20503 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:31:54.038537 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-47ffd843-b579-48dd-9e5a-85e58ca20503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:31:54.110655 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ffd843-b579-48dd-9e5a-85e58ca20503 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:31:54.114514 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad96ea-85e9-42fb-8ee1-0144a1b2af52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:31:54.115196 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad96ea-85e9-42fb-8ee1-0144a1b2af52 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:31:54.115504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad96ea-85e9-42fb-8ee1-0144a1b2af52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:31:54.115892 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad96ea-85e9-42fb-8ee1-0144a1b2af52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:31:54.126509 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad96ea-85e9-42fb-8ee1-0144a1b2af52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:31:54.128010 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad96ea-85e9-42fb-8ee1-0144a1b2af52 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:31:54.133365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:31:54.138379 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 768/1530] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:31:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:31:56.310513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05a24f-180a-40e3-8e5e-9bfb795dd558 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:31:56.311156 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05a24f-180a-40e3-8e5e-9bfb795dd558 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:31:56.311156 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05a24f-180a-40e3-8e5e-9bfb795dd558 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:31:56.312613 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 763/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 16:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:31:56.318664 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726ac99-8d3b-4cb0-9294-5cf7c532f749 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:31:56.319530 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726ac99-8d3b-4cb0-9294-5cf7c532f749 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:31:56.319530 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726ac99-8d3b-4cb0-9294-5cf7c532f749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:31:56.340869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c726ac99-8d3b-4cb0-9294-5cf7c532f749 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:31:56.342129 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c726ac99-8d3b-4cb0-9294-5cf7c532f749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:31:56.351688 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 769/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 16:31:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:31:56.392359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05a24f-180a-40e3-8e5e-9bfb795dd558 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:31:56.395524 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6f05a24f-180a-40e3-8e5e-9bfb795dd558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:31:56.418396 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05a24f-180a-40e3-8e5e-9bfb795dd558 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:31:56.425245 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7aa4f-0b7b-4f60-a1d2-b4f0084d66cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:31:56.426378 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7aa4f-0b7b-4f60-a1d2-b4f0084d66cd service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:31:56.426884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7aa4f-0b7b-4f60-a1d2-b4f0084d66cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:31:56.427375 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7aa4f-0b7b-4f60-a1d2-b4f0084d66cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:31:56.444234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7aa4f-0b7b-4f60-a1d2-b4f0084d66cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:31:56.445745 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7aa4f-0b7b-4f60-a1d2-b4f0084d66cd service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:31:56.458393 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 764/1533] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:31:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:32:00.354522 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88b91a-da41-4ae5-9960-ace7536f028e None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:32:00.355123 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88b91a-da41-4ae5-9960-ace7536f028e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:32:00.355123 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88b91a-da41-4ae5-9960-ace7536f028e None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:32:00.356761 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 770/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 16:32:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:32:00.364648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17581285-8e79-419c-8896-49627c305617 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:32:00.365471 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17581285-8e79-419c-8896-49627c305617 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:32:00.365471 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17581285-8e79-419c-8896-49627c305617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:32:00.386012 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-17581285-8e79-419c-8896-49627c305617 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:32:00.387296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-17581285-8e79-419c-8896-49627c305617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:32:00.396983 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 765/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 16:32:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:32:00.724872 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88b91a-da41-4ae5-9960-ace7536f028e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:32:00.726611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4b88b91a-da41-4ae5-9960-ace7536f028e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:32:00.790382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88b91a-da41-4ae5-9960-ace7536f028e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:32:00.794194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf3692-253a-4c1c-854f-e23d50863aca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:32:00.794857 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf3692-253a-4c1c-854f-e23d50863aca service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:32:00.795147 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf3692-253a-4c1c-854f-e23d50863aca service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:32:00.795427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf3692-253a-4c1c-854f-e23d50863aca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:32:00.806040 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf3692-253a-4c1c-854f-e23d50863aca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:32:00.807829 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf3692-253a-4c1c-854f-e23d50863aca service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:32:00.820175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 771/1536] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:32:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:32:03.008031 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc0ae6-fcd9-4731-a4f9-7935dbcf765e None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:32:03.008875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc0ae6-fcd9-4731-a4f9-7935dbcf765e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:32:03.009063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dc0ae6-fcd9-4731-a4f9-7935dbcf765e None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:32:03.010678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 766/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 16:32:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:32:03.016378 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e2a5a-cd89-4b63-b4cb-f67be554941f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:32:03.016885 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e2a5a-cd89-4b63-b4cb-f67be554941f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:32:03.016981 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377e2a5a-cd89-4b63-b4cb-f67be554941f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:32:03.038207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-377e2a5a-cd89-4b63-b4cb-f67be554941f None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:32:03.039376 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-377e2a5a-cd89-4b63-b4cb-f67be554941f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:32:03.048641 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 772/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 16:32:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:32:03.370168 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc0ae6-fcd9-4731-a4f9-7935dbcf765e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:32:03.371980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-27dc0ae6-fcd9-4731-a4f9-7935dbcf765e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:32:03.385740 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dc0ae6-fcd9-4731-a4f9-7935dbcf765e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:32:03.389567 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eea092-1e50-4ddf-b706-d18bb9aa9481 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:32:03.390255 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eea092-1e50-4ddf-b706-d18bb9aa9481 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:32:03.391034 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eea092-1e50-4ddf-b706-d18bb9aa9481 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:32:03.391034 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eea092-1e50-4ddf-b706-d18bb9aa9481 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:32:03.401124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eea092-1e50-4ddf-b706-d18bb9aa9481 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:32:03.402571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eea092-1e50-4ddf-b706-d18bb9aa9481 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:32:03.415335 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 767/1539] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:32:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:32:13.030465 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=32 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:32:13.410595 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=70 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:33:33.969474 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88e113-bc94-40e4-adf0-7c476238ebb0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:33.970864 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88e113-bc94-40e4-adf0-7c476238ebb0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:33.971405 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88e113-bc94-40e4-adf0-7c476238ebb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:33.998563 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6c88e113-bc94-40e4-adf0-7c476238ebb0 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:34.000902 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6c88e113-bc94-40e4-adf0-7c476238ebb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:34.012857 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 773/1540] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:34.024529 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f073c4d-db40-441d-af2c-7b83f4160c7b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:34.025128 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f073c4d-db40-441d-af2c-7b83f4160c7b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:34.025514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f073c4d-db40-441d-af2c-7b83f4160c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:34.063905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0f073c4d-db40-441d-af2c-7b83f4160c7b None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:34.065246 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0f073c4d-db40-441d-af2c-7b83f4160c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:34.079730 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 768/1541] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:33:34.085151 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c88e113-bc94-40e4-adf0-7c476238ebb0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:34.086743 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f536fe-0d30-43c8-8422-db12d3f5a939 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:34.087032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f536fe-0d30-43c8-8422-db12d3f5a939 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:34.087326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f536fe-0d30-43c8-8422-db12d3f5a939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:34.102303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c88e113-bc94-40e4-adf0-7c476238ebb0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:34.117078 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb9708-9202-4011-8f41-51f09b5b27b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:34.118308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb9708-9202-4011-8f41-51f09b5b27b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:34.118973 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb9708-9202-4011-8f41-51f09b5b27b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:34.119511 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb9708-9202-4011-8f41-51f09b5b27b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:34.120783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb9708-9202-4011-8f41-51f09b5b27b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:33:34.122012 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb9708-9202-4011-8f41-51f09b5b27b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:34.123974 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb9708-9202-4011-8f41-51f09b5b27b3 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:34.124689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-40f536fe-0d30-43c8-8422-db12d3f5a939 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:34.125928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-40f536fe-0d30-43c8-8422-db12d3f5a939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:34.140208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 769/1542] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:33:34.149848 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 774/1543] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:33:34.158444 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9fa66-87f5-4390-83d9-d92a325f8ec5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:34.158946 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9fa66-87f5-4390-83d9-d92a325f8ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:34.159453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9fa66-87f5-4390-83d9-d92a325f8ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:34.163628 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb9708-9202-4011-8f41-51f09b5b27b3 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:34.202622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb9708-9202-4011-8f41-51f09b5b27b3 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:34.229803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940371f8-e605-4c14-b62b-1388eb7b3acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:34.231461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940371f8-e605-4c14-b62b-1388eb7b3acb admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:34.232063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940371f8-e605-4c14-b62b-1388eb7b3acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:34.232529 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940371f8-e605-4c14-b62b-1388eb7b3acb admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:34.233902 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940371f8-e605-4c14-b62b-1388eb7b3acb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:34.239144 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1cc9fa66-87f5-4390-83d9-d92a325f8ec5 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:34.240409 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: WARNING py.warnings [None req-940371f8-e605-4c14-b62b-1388eb7b3acb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 16:33:34.240409 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: warnings.warn(msg) Jun 30 16:33:34.240409 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: Jun 30 16:33:34.240709 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1cc9fa66-87f5-4390-83d9-d92a325f8ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:34.243055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940371f8-e605-4c14-b62b-1388eb7b3acb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:34.267589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 770/1544] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:34.273223 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc9fa66-87f5-4390-83d9-d92a325f8ec5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:34.295030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc9fa66-87f5-4390-83d9-d92a325f8ec5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:34.307058 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c7bc68-30f2-4bee-a135-7ee8cbc31a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:34.307852 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7bc68-30f2-4bee-a135-7ee8cbc31a0a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:34.308134 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7bc68-30f2-4bee-a135-7ee8cbc31a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:34.309413 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7bc68-30f2-4bee-a135-7ee8cbc31a0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:34.310215 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c7bc68-30f2-4bee-a135-7ee8cbc31a0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:33:34.310561 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c7bc68-30f2-4bee-a135-7ee8cbc31a0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:34.311633 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-f4c7bc68-30f2-4bee-a135-7ee8cbc31a0a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 16:33:34.311633 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Jun 30 16:33:34.311633 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: Jun 30 16:33:34.312545 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c7bc68-30f2-4bee-a135-7ee8cbc31a0a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:34.331601 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 771/1545] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:33:34.333025 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b44de-5f43-4929-ad93-7dbd1d96ba88 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:34.333318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b44de-5f43-4929-ad93-7dbd1d96ba88 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:34.333603 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b44de-5f43-4929-ad93-7dbd1d96ba88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:34.372600 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-013b44de-5f43-4929-ad93-7dbd1d96ba88 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:34.374254 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-013b44de-5f43-4929-ad93-7dbd1d96ba88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:34.390144 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 772/1546] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:34.394469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b44de-5f43-4929-ad93-7dbd1d96ba88 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:34.398844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 775/1547] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:33:34 2022] POST /identity/v3/projects => generated 379 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:33:34.404592 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940371f8-e605-4c14-b62b-1388eb7b3acb admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:34.408159 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b44de-5f43-4929-ad93-7dbd1d96ba88 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:34.550220 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940371f8-e605-4c14-b62b-1388eb7b3acb admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:34.564951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0562310-1396-493f-aaaa-0bcc58d48390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:34.565795 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0562310-1396-493f-aaaa-0bcc58d48390 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:34.566085 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0562310-1396-493f-aaaa-0bcc58d48390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:34.566361 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0562310-1396-493f-aaaa-0bcc58d48390 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:34.567062 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0562310-1396-493f-aaaa-0bcc58d48390 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:34.569181 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0562310-1396-493f-aaaa-0bcc58d48390 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:34.570778 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e26113-171f-47f1-8f5e-128274c65d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:34.571496 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e26113-171f-47f1-8f5e-128274c65d26 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:34.571789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e26113-171f-47f1-8f5e-128274c65d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:34.572066 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e26113-171f-47f1-8f5e-128274c65d26 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:34.573222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e26113-171f-47f1-8f5e-128274c65d26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:34.576089 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e26113-171f-47f1-8f5e-128274c65d26 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:34.666511 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 776/1548] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:33:34 2022] POST /identity/v3/users => generated 290 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:33:34.668316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a22ed9-0b4f-46a2-85e1-917338c6c2fa None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:34.668827 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a22ed9-0b4f-46a2-85e1-917338c6c2fa None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:34.669297 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a22ed9-0b4f-46a2-85e1-917338c6c2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:34.701781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 773/1549] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:33:34 2022] POST /identity/v3/projects => generated 393 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:33:34.702961 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-17a22ed9-0b4f-46a2-85e1-917338c6c2fa None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:34.703817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5477cfa-a325-4d52-9115-4dd7913ef518 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:34.704107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5477cfa-a325-4d52-9115-4dd7913ef518 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:34.704231 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-17a22ed9-0b4f-46a2-85e1-917338c6c2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:34.704428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5477cfa-a325-4d52-9115-4dd7913ef518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:34.743892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b5477cfa-a325-4d52-9115-4dd7913ef518 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:34.745255 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b5477cfa-a325-4d52-9115-4dd7913ef518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:34.814840 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 777/1550] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:34.816040 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a22ed9-0b4f-46a2-85e1-917338c6c2fa None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:34.831038 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a22ed9-0b4f-46a2-85e1-917338c6c2fa None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:34.836262 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8285233-036a-4131-8c26-bb2b254b9fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:34.836930 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8285233-036a-4131-8c26-bb2b254b9fbc admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:34.837453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8285233-036a-4131-8c26-bb2b254b9fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:34.837453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8285233-036a-4131-8c26-bb2b254b9fbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:34.838205 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8285233-036a-4131-8c26-bb2b254b9fbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:34.840041 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8285233-036a-4131-8c26-bb2b254b9fbc admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:34.848638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 774/1551] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:34.849459 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5477cfa-a325-4d52-9115-4dd7913ef518 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:34.854763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 778/1552] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:33:34.854763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f833cc9-d96a-405d-aa93-e72105305aac None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:34.854763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f833cc9-d96a-405d-aa93-e72105305aac None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:34.854763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f833cc9-d96a-405d-aa93-e72105305aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:34.861591 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5477cfa-a325-4d52-9115-4dd7913ef518 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:34.866253 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb71736-152e-42ce-b97f-c84dde7abddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:34.866950 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb71736-152e-42ce-b97f-c84dde7abddf admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:34.867229 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb71736-152e-42ce-b97f-c84dde7abddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:34.867528 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb71736-152e-42ce-b97f-c84dde7abddf admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:34.868218 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb71736-152e-42ce-b97f-c84dde7abddf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:34.869252 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-7cb71736-152e-42ce-b97f-c84dde7abddf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 16:33:34.869252 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Jun 30 16:33:34.869252 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: Jun 30 16:33:34.870463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb71736-152e-42ce-b97f-c84dde7abddf admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:34.873288 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2f833cc9-d96a-405d-aa93-e72105305aac None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:34.874490 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2f833cc9-d96a-405d-aa93-e72105305aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:34.882769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 779/1553] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:34.884221 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ceeba8-48f0-4b4c-9f72-4e2b44db83c8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:34.884596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ceeba8-48f0-4b4c-9f72-4e2b44db83c8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:34.884979 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ceeba8-48f0-4b4c-9f72-4e2b44db83c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:34.904498 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b7ceeba8-48f0-4b4c-9f72-4e2b44db83c8 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:34.905779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b7ceeba8-48f0-4b4c-9f72-4e2b44db83c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:34.913669 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 780/1554] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:34.916750 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077b448-09cf-4b34-ba53-af7315cc21eb None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:34.916750 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077b448-09cf-4b34-ba53-af7315cc21eb None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:34.916750 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077b448-09cf-4b34-ba53-af7315cc21eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:34.935263 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1077b448-09cf-4b34-ba53-af7315cc21eb None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:34.936523 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1077b448-09cf-4b34-ba53-af7315cc21eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:34.944647 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 781/1555] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:34.945498 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077b448-09cf-4b34-ba53-af7315cc21eb None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:34.955814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077b448-09cf-4b34-ba53-af7315cc21eb None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:34.960701 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f25b3f-90bd-48ad-a613-d70b2c018ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:34.961381 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f25b3f-90bd-48ad-a613-d70b2c018ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:34.961662 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f25b3f-90bd-48ad-a613-d70b2c018ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:34.961966 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f25b3f-90bd-48ad-a613-d70b2c018ae8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:34.962701 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f25b3f-90bd-48ad-a613-d70b2c018ae8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:33:34.963008 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f25b3f-90bd-48ad-a613-d70b2c018ae8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:34.964698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f25b3f-90bd-48ad-a613-d70b2c018ae8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:34.978749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 782/1556] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:33:34.980783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c7958-442f-41bf-b00d-a3112b029f81 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:34.981393 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c7958-442f-41bf-b00d-a3112b029f81 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:34.981643 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c7958-442f-41bf-b00d-a3112b029f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:34.989166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:33:35.001536 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-057c7958-442f-41bf-b00d-a3112b029f81 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:35.002795 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-057c7958-442f-41bf-b00d-a3112b029f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:35.011019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 783/1557] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:35.018569 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c7958-442f-41bf-b00d-a3112b029f81 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:35.022907 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c7958-442f-41bf-b00d-a3112b029f81 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:35.027666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b1bf0-097c-433d-93a4-ca2ff5f965fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:35.030077 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b1bf0-097c-433d-93a4-ca2ff5f965fd admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:35.030077 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b1bf0-097c-433d-93a4-ca2ff5f965fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:35.030077 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b1bf0-097c-433d-93a4-ca2ff5f965fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:35.030077 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71b1bf0-097c-433d-93a4-ca2ff5f965fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:35.031518 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71b1bf0-097c-433d-93a4-ca2ff5f965fd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:35.042013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 784/1558] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:33:35.044380 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081d491-733a-45d6-ac07-1cd6ee51db4e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:35.044674 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081d491-733a-45d6-ac07-1cd6ee51db4e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:35.044954 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f081d491-733a-45d6-ac07-1cd6ee51db4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:35.065004 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f081d491-733a-45d6-ac07-1cd6ee51db4e None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:35.066238 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f081d491-733a-45d6-ac07-1cd6ee51db4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:35.074679 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 785/1559] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:35.078509 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bff2ec-98e4-4928-9aab-f50113de49fb None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:35.078509 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bff2ec-98e4-4928-9aab-f50113de49fb None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:35.078509 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bff2ec-98e4-4928-9aab-f50113de49fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:35.096282 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-84bff2ec-98e4-4928-9aab-f50113de49fb None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:35.097512 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-84bff2ec-98e4-4928-9aab-f50113de49fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:35.106406 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 786/1560] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:35.108727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a07ba1-d270-4180-a9da-f1e282bdef32 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:35.109231 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a07ba1-d270-4180-a9da-f1e282bdef32 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:35.109718 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a07ba1-d270-4180-a9da-f1e282bdef32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:35.131969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-18a07ba1-d270-4180-a9da-f1e282bdef32 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:35.133094 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 775/1561] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:33:34 2022] POST /identity/v3/users => generated 297 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:33:35.133594 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-18a07ba1-d270-4180-a9da-f1e282bdef32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:35.134341 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ce5db-0789-4df3-ab30-e5abb46e3f1a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:35.134617 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ce5db-0789-4df3-ab30-e5abb46e3f1a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:35.135057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ce5db-0789-4df3-ab30-e5abb46e3f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:35.143400 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 787/1562] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:35.144377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a07ba1-d270-4180-a9da-f1e282bdef32 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:35.156315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a07ba1-d270-4180-a9da-f1e282bdef32 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:35.156600 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-319ce5db-0789-4df3-ab30-e5abb46e3f1a None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:35.157811 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-319ce5db-0789-4df3-ab30-e5abb46e3f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:35.165117 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d5976-d4a1-43f4-8399-bab340cbedcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:35.165117 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d5976-d4a1-43f4-8399-bab340cbedcd admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:35.165117 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d5976-d4a1-43f4-8399-bab340cbedcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:35.165117 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d5976-d4a1-43f4-8399-bab340cbedcd admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:35.165643 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d5976-d4a1-43f4-8399-bab340cbedcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:35.165643 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9d5976-d4a1-43f4-8399-bab340cbedcd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:35.171787 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 776/1563] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:35.173265 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36a3a0-5056-4ac7-9a2f-1d4804b2624f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:35.173703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36a3a0-5056-4ac7-9a2f-1d4804b2624f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:35.173991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36a3a0-5056-4ac7-9a2f-1d4804b2624f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:35.194429 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4a36a3a0-5056-4ac7-9a2f-1d4804b2624f None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:35.198493 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4a36a3a0-5056-4ac7-9a2f-1d4804b2624f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:35.203890 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 777/1564] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:35.204713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a36a3a0-5056-4ac7-9a2f-1d4804b2624f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:35.214523 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a36a3a0-5056-4ac7-9a2f-1d4804b2624f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:35.219628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ea430-8c13-4364-a840-68e12eb9d11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:35.220531 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ea430-8c13-4364-a840-68e12eb9d11d admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:35.220811 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ea430-8c13-4364-a840-68e12eb9d11d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:35.221086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ea430-8c13-4364-a840-68e12eb9d11d admin admin] PATH_INFO: `/v3/projects/2da3dc7978d5414cba40e9e08dc39d94/users/5703438f487a45abbc5b68b77519534c/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:35.233905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:33:35.237036 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ea430-8c13-4364-a840-68e12eb9d11d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2da3dc7978d5414cba40e9e08dc39d94, user_id=5703438f487a45abbc5b68b77519534c, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:35.239747 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ea430-8c13-4364-a840-68e12eb9d11d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:35.258991 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 788/1565] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:33:35 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:33:35.259858 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d5976-d4a1-43f4-8399-bab340cbedcd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:35.322440 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d5976-d4a1-43f4-8399-bab340cbedcd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:35.327489 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef30b20-896c-492a-85bf-9779d94261c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:35.328637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef30b20-896c-492a-85bf-9779d94261c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:35.328637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef30b20-896c-492a-85bf-9779d94261c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:35.328816 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef30b20-896c-492a-85bf-9779d94261c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:35.329484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef30b20-896c-492a-85bf-9779d94261c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:33:35.329790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef30b20-896c-492a-85bf-9779d94261c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:35.330577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 778/1566] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:33:35 2022] PUT /identity/v3/projects/2da3dc7978d5414cba40e9e08dc39d94/users/5703438f487a45abbc5b68b77519534c/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:33:35.331404 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef30b20-896c-492a-85bf-9779d94261c3 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:35.331556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ea430-8c13-4364-a840-68e12eb9d11d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:35.344836 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 789/1567] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:33:35.345725 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef30b20-896c-492a-85bf-9779d94261c3 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:35.360892 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef30b20-896c-492a-85bf-9779d94261c3 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:35.397849 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ea430-8c13-4364-a840-68e12eb9d11d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:35.403544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f742182-d790-474f-bd93-70793992a718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:35.404299 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f742182-d790-474f-bd93-70793992a718 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:35.404602 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f742182-d790-474f-bd93-70793992a718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:35.404880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f742182-d790-474f-bd93-70793992a718 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:35.405602 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f742182-d790-474f-bd93-70793992a718 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:33:35.405918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f742182-d790-474f-bd93-70793992a718 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:35.407551 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f742182-d790-474f-bd93-70793992a718 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:35.419844 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 779/1568] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:33:35.422262 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c1d2c-10ba-43b4-a6d4-7abd8b57c649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:35.427433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c1d2c-10ba-43b4-a6d4-7abd8b57c649 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:35.427433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c1d2c-10ba-43b4-a6d4-7abd8b57c649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:35.427433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c1d2c-10ba-43b4-a6d4-7abd8b57c649 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:35.427433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c1d2c-10ba-43b4-a6d4-7abd8b57c649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:35.427433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c1d2c-10ba-43b4-a6d4-7abd8b57c649 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:35.427938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f742182-d790-474f-bd93-70793992a718 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:35.437162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 790/1569] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:33:35.437578 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f742182-d790-474f-bd93-70793992a718 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:35.437995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c1d2c-10ba-43b4-a6d4-7abd8b57c649 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:35.443128 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3b55f-492c-4adf-86bd-88bd3ee01149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:35.443834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3b55f-492c-4adf-86bd-88bd3ee01149 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:35.444123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3b55f-492c-4adf-86bd-88bd3ee01149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:35.444402 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3b55f-492c-4adf-86bd-88bd3ee01149 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:35.445220 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3b55f-492c-4adf-86bd-88bd3ee01149 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:33:35.445538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3b55f-492c-4adf-86bd-88bd3ee01149 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:35.447150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3b55f-492c-4adf-86bd-88bd3ee01149 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:35.448310 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c1d2c-10ba-43b4-a6d4-7abd8b57c649 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:35.453036 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85dfd25-5aa9-43ac-bfe0-5f2c57f74ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:35.468365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85dfd25-5aa9-43ac-bfe0-5f2c57f74ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:35.468365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85dfd25-5aa9-43ac-bfe0-5f2c57f74ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:35.468365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85dfd25-5aa9-43ac-bfe0-5f2c57f74ee1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:35.468365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85dfd25-5aa9-43ac-bfe0-5f2c57f74ee1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:33:35.468365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85dfd25-5aa9-43ac-bfe0-5f2c57f74ee1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:35.468365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85dfd25-5aa9-43ac-bfe0-5f2c57f74ee1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:35.468961 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 780/1570] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:33:35.468961 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3b55f-492c-4adf-86bd-88bd3ee01149 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:35.475534 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 791/1571] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:33:35.475534 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85dfd25-5aa9-43ac-bfe0-5f2c57f74ee1 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:35.482385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3b55f-492c-4adf-86bd-88bd3ee01149 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:35.482661 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85dfd25-5aa9-43ac-bfe0-5f2c57f74ee1 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:35.487044 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e03626-a5be-414b-8423-13d8498cb2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:35.487225 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198fe075-811b-4ced-86ca-763146bf7564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:35.487749 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e03626-a5be-414b-8423-13d8498cb2cf admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:35.487931 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198fe075-811b-4ced-86ca-763146bf7564 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:35.488051 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e03626-a5be-414b-8423-13d8498cb2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:35.488241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198fe075-811b-4ced-86ca-763146bf7564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:35.488340 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e03626-a5be-414b-8423-13d8498cb2cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:35.488517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198fe075-811b-4ced-86ca-763146bf7564 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:35.489017 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e03626-a5be-414b-8423-13d8498cb2cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:33:35.489200 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198fe075-811b-4ced-86ca-763146bf7564 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:35.489328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e03626-a5be-414b-8423-13d8498cb2cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:35.490915 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e03626-a5be-414b-8423-13d8498cb2cf admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:35.491170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198fe075-811b-4ced-86ca-763146bf7564 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:35.503209 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 781/1572] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:33:35.504521 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74d873-8c39-4c87-bc75-19fd6880805e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:35.504812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74d873-8c39-4c87-bc75-19fd6880805e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:35.505090 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74d873-8c39-4c87-bc75-19fd6880805e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:35.523152 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6f74d873-8c39-4c87-bc75-19fd6880805e None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:35.524464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6f74d873-8c39-4c87-bc75-19fd6880805e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:35.533168 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 782/1573] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:35.533911 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74d873-8c39-4c87-bc75-19fd6880805e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:35.544648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f74d873-8c39-4c87-bc75-19fd6880805e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:35.549260 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d3c57f-626c-4d10-ae16-013bbe61b9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:35.549905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3c57f-626c-4d10-ae16-013bbe61b9ce admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:35.550183 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3c57f-626c-4d10-ae16-013bbe61b9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:35.550469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d3c57f-626c-4d10-ae16-013bbe61b9ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:35.551131 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d3c57f-626c-4d10-ae16-013bbe61b9ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:35.553038 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d3c57f-626c-4d10-ae16-013bbe61b9ce admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:35.586538 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 792/1574] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:33:35 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:33:35.586538 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198fe075-811b-4ced-86ca-763146bf7564 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:35.591480 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:33:35.595260 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198fe075-811b-4ced-86ca-763146bf7564 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:35.599781 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7563d2-ee55-44d7-a8f3-7aca0921b9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:35.600420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7563d2-ee55-44d7-a8f3-7aca0921b9f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:35.600707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7563d2-ee55-44d7-a8f3-7aca0921b9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:35.600986 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7563d2-ee55-44d7-a8f3-7aca0921b9f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:35.601685 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7563d2-ee55-44d7-a8f3-7aca0921b9f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:35.603609 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7563d2-ee55-44d7-a8f3-7aca0921b9f2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:35.664327 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 783/1575] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:33:35 2022] POST /identity/v3/projects => generated 383 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:33:35.665805 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d3c57f-626c-4d10-ae16-013bbe61b9ce admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:35.689019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 793/1576] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:33:35 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:33:35.694133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7563d2-ee55-44d7-a8f3-7aca0921b9f2 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:35.740385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d3c57f-626c-4d10-ae16-013bbe61b9ce admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:35.769828 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7563d2-ee55-44d7-a8f3-7aca0921b9f2 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:35.774563 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf32ea-1255-4b87-bb37-fb0b26dc8903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:35.775234 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf32ea-1255-4b87-bb37-fb0b26dc8903 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:35.775543 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf32ea-1255-4b87-bb37-fb0b26dc8903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:35.775824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf32ea-1255-4b87-bb37-fb0b26dc8903 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:35.776535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf32ea-1255-4b87-bb37-fb0b26dc8903 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:35.779342 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf32ea-1255-4b87-bb37-fb0b26dc8903 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:35.780939 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:33:35.805288 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1f8a4-a3e0-4a6c-b118-d6cb8bbc10e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:35.805969 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1f8a4-a3e0-4a6c-b118-d6cb8bbc10e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:35.806258 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1f8a4-a3e0-4a6c-b118-d6cb8bbc10e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:35.806537 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e1f8a4-a3e0-4a6c-b118-d6cb8bbc10e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:35.807263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1f8a4-a3e0-4a6c-b118-d6cb8bbc10e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:35.809145 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e1f8a4-a3e0-4a6c-b118-d6cb8bbc10e4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:35.819869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 784/1577] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:33:35.820655 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1f8a4-a3e0-4a6c-b118-d6cb8bbc10e4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:35.831150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e1f8a4-a3e0-4a6c-b118-d6cb8bbc10e4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:35.835808 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5128b8-8441-405b-b234-e2024765dcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:35.836457 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5128b8-8441-405b-b234-e2024765dcf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:35.836797 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5128b8-8441-405b-b234-e2024765dcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:35.837024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5128b8-8441-405b-b234-e2024765dcf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:35.837714 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5128b8-8441-405b-b234-e2024765dcf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:35.840317 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5128b8-8441-405b-b234-e2024765dcf2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:35.910249 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 794/1578] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:33:35 2022] POST /identity/v3/projects => generated 375 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:33:35.911299 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf32ea-1255-4b87-bb37-fb0b26dc8903 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:35.946964 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 785/1579] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:33:35 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:33:35.947883 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5128b8-8441-405b-b234-e2024765dcf2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:35.978016 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf32ea-1255-4b87-bb37-fb0b26dc8903 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:36.008522 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5128b8-8441-405b-b234-e2024765dcf2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:36.013110 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff010f9-be94-49b5-827d-e2c5ef95a40d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:36.013767 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff010f9-be94-49b5-827d-e2c5ef95a40d service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:36.014051 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff010f9-be94-49b5-827d-e2c5ef95a40d service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:36.014332 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff010f9-be94-49b5-827d-e2c5ef95a40d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:36.036226 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f063e-54d0-4a59-b1d6-580086dda047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:36.036891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f063e-54d0-4a59-b1d6-580086dda047 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:36.037182 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f063e-54d0-4a59-b1d6-580086dda047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:36.037461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f063e-54d0-4a59-b1d6-580086dda047 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:36.038179 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f063e-54d0-4a59-b1d6-580086dda047 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:36.040153 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f063e-54d0-4a59-b1d6-580086dda047 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:36.077098 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff010f9-be94-49b5-827d-e2c5ef95a40d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:36.078589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff010f9-be94-49b5-827d-e2c5ef95a40d service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:36.088193 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 786/1580] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:33:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:33:36.088926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff010f9-be94-49b5-827d-e2c5ef95a40d service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:36.098834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff010f9-be94-49b5-827d-e2c5ef95a40d service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:36.103489 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1973461-bf57-497c-ac2e-12bdb2bd8f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:36.104156 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1973461-bf57-497c-ac2e-12bdb2bd8f18 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:36.104434 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1973461-bf57-497c-ac2e-12bdb2bd8f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:36.105043 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1973461-bf57-497c-ac2e-12bdb2bd8f18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:36.105651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1973461-bf57-497c-ac2e-12bdb2bd8f18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:36.107269 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1973461-bf57-497c-ac2e-12bdb2bd8f18 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:36.117773 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 787/1581] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:33:36.118728 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1973461-bf57-497c-ac2e-12bdb2bd8f18 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:36.127780 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:33:36.130328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1973461-bf57-497c-ac2e-12bdb2bd8f18 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:36.134559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 795/1582] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:33:35 2022] POST /identity/v3/projects => generated 385 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:33:36.135187 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f063e-54d0-4a59-b1d6-580086dda047 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:36.141389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3da10-b5d0-45f7-a7c3-ab34eb5919f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:36.142735 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3da10-b5d0-45f7-a7c3-ab34eb5919f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:36.143189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3da10-b5d0-45f7-a7c3-ab34eb5919f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:36.143689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3da10-b5d0-45f7-a7c3-ab34eb5919f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:36.144850 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e3da10-b5d0-45f7-a7c3-ab34eb5919f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:36.148227 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e3da10-b5d0-45f7-a7c3-ab34eb5919f6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:36.212970 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f063e-54d0-4a59-b1d6-580086dda047 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:36.218321 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee40804-9897-4749-a318-4e4dae01ff1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:36.219086 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee40804-9897-4749-a318-4e4dae01ff1d admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:36.219372 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee40804-9897-4749-a318-4e4dae01ff1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:36.219690 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee40804-9897-4749-a318-4e4dae01ff1d admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:36.221031 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee40804-9897-4749-a318-4e4dae01ff1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:36.222567 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee40804-9897-4749-a318-4e4dae01ff1d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:36.288746 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 788/1583] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:33:36 2022] POST /identity/v3/users => generated 292 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:33:36.289796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3da10-b5d0-45f7-a7c3-ab34eb5919f6 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:36.301335 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3da10-b5d0-45f7-a7c3-ab34eb5919f6 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:36.306235 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb2d50-33e1-4963-b487-4a8c0dd566c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:36.307137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb2d50-33e1-4963-b487-4a8c0dd566c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:36.307421 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb2d50-33e1-4963-b487-4a8c0dd566c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:36.307750 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb2d50-33e1-4963-b487-4a8c0dd566c0 admin admin] PATH_INFO: `/v3/projects/839a189179c543c6a704a968170ec022/users/2de23770f4374decb95ff94f3e7f793b/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:36.319875 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 796/1584] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:33:36 2022] POST /identity/v3/users => generated 291 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:33:36.322023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee40804-9897-4749-a318-4e4dae01ff1d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:36.329767 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb2d50-33e1-4963-b487-4a8c0dd566c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=839a189179c543c6a704a968170ec022, user_id=2de23770f4374decb95ff94f3e7f793b, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:36.329767 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb2d50-33e1-4963-b487-4a8c0dd566c0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:36.336115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee40804-9897-4749-a318-4e4dae01ff1d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:36.341916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b1561-b624-4190-a190-aec0b5606603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:36.342529 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b1561-b624-4190-a190-aec0b5606603 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:36.342865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b1561-b624-4190-a190-aec0b5606603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:36.343151 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b1561-b624-4190-a190-aec0b5606603 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:36.343908 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b1561-b624-4190-a190-aec0b5606603 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:36.345972 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b1561-b624-4190-a190-aec0b5606603 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:36.478188 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 789/1585] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:33:36 2022] PUT /identity/v3/projects/839a189179c543c6a704a968170ec022/users/2de23770f4374decb95ff94f3e7f793b/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:33:36.479102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb2d50-33e1-4963-b487-4a8c0dd566c0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:36.494422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:33:36.505545 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 797/1586] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:33:36 2022] POST /identity/v3/users => generated 288 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:33:36.505545 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b1561-b624-4190-a190-aec0b5606603 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:36.515655 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b1561-b624-4190-a190-aec0b5606603 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:36.562012 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb2d50-33e1-4963-b487-4a8c0dd566c0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:36.566746 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159affc-5d42-498f-83ac-ea301a2d6e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:36.567427 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159affc-5d42-498f-83ac-ea301a2d6e5a admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:36.567739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159affc-5d42-498f-83ac-ea301a2d6e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:36.568020 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159affc-5d42-498f-83ac-ea301a2d6e5a admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:36.568724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159affc-5d42-498f-83ac-ea301a2d6e5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:36.570694 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159affc-5d42-498f-83ac-ea301a2d6e5a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:36.585826 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bd391-27ce-4b18-a0c7-5ca0a4a9ceba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:36.587903 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bd391-27ce-4b18-a0c7-5ca0a4a9ceba admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:36.587903 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bd391-27ce-4b18-a0c7-5ca0a4a9ceba admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:36.587903 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0bd391-27ce-4b18-a0c7-5ca0a4a9ceba admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:36.587903 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bd391-27ce-4b18-a0c7-5ca0a4a9ceba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:36.589544 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0bd391-27ce-4b18-a0c7-5ca0a4a9ceba admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:36.599941 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 798/1587] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:33:36.600652 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bd391-27ce-4b18-a0c7-5ca0a4a9ceba admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:36.610686 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0bd391-27ce-4b18-a0c7-5ca0a4a9ceba admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:36.615258 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8925491-682d-49ab-8ffb-2e5b89ae95a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:36.616643 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8925491-682d-49ab-8ffb-2e5b89ae95a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:36.616643 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8925491-682d-49ab-8ffb-2e5b89ae95a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:36.616643 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8925491-682d-49ab-8ffb-2e5b89ae95a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:36.617154 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8925491-682d-49ab-8ffb-2e5b89ae95a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:36.619072 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8925491-682d-49ab-8ffb-2e5b89ae95a3 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:36.677343 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 790/1588] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:33:36 2022] POST /identity/v3/users => generated 287 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:33:36.679617 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159affc-5d42-498f-83ac-ea301a2d6e5a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:36.692884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159affc-5d42-498f-83ac-ea301a2d6e5a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:36.698840 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33050e6-bc32-4521-b32c-04d8d18f90ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:36.698840 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33050e6-bc32-4521-b32c-04d8d18f90ef admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:36.698840 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33050e6-bc32-4521-b32c-04d8d18f90ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:36.699657 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33050e6-bc32-4521-b32c-04d8d18f90ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:36.700612 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33050e6-bc32-4521-b32c-04d8d18f90ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:36.702269 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33050e6-bc32-4521-b32c-04d8d18f90ef admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:36.713455 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 791/1589] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:33:36.714325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33050e6-bc32-4521-b32c-04d8d18f90ef admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:36.726506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33050e6-bc32-4521-b32c-04d8d18f90ef admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:36.731915 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb94036-2e91-4266-a71c-35414c2527bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:36.732609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb94036-2e91-4266-a71c-35414c2527bc admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:36.732876 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb94036-2e91-4266-a71c-35414c2527bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:36.733153 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb94036-2e91-4266-a71c-35414c2527bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:36.733877 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb94036-2e91-4266-a71c-35414c2527bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:36.735508 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb94036-2e91-4266-a71c-35414c2527bc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:36.749252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 799/1590] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:33:36 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:33:36.749589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 792/1591] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:33:36.749687 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1d31c-5fff-4d21-8114-c2998cd52248 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:36.749797 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb94036-2e91-4266-a71c-35414c2527bc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:36.749877 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1d31c-5fff-4d21-8114-c2998cd52248 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:36.749877 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1d31c-5fff-4d21-8114-c2998cd52248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:36.764461 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb94036-2e91-4266-a71c-35414c2527bc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:36.772210 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8680a01-6206-42e6-8019-9a6a2680c657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:36.772210 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8680a01-6206-42e6-8019-9a6a2680c657 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:36.772210 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8680a01-6206-42e6-8019-9a6a2680c657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:36.772210 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8680a01-6206-42e6-8019-9a6a2680c657 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:36.773860 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8680a01-6206-42e6-8019-9a6a2680c657 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:36.775256 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8680a01-6206-42e6-8019-9a6a2680c657 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:36.793410 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 793/1592] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:33:36.794501 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8680a01-6206-42e6-8019-9a6a2680c657 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:36.806229 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8680a01-6206-42e6-8019-9a6a2680c657 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:36.813174 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7038607-70ca-41b0-bab5-08ca2ce6d5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:36.821185 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7038607-70ca-41b0-bab5-08ca2ce6d5b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:36.821185 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7038607-70ca-41b0-bab5-08ca2ce6d5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:36.821185 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7038607-70ca-41b0-bab5-08ca2ce6d5b0 admin admin] PATH_INFO: `/v3/projects/f942ab31bafc428bb1fba94aee2e0c32/users/0a26028451b345968fe86774f44736dd/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:36.841764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7038607-70ca-41b0-bab5-08ca2ce6d5b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f942ab31bafc428bb1fba94aee2e0c32, user_id=0a26028451b345968fe86774f44736dd, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:36.845995 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7038607-70ca-41b0-bab5-08ca2ce6d5b0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:36.926654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 794/1593] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:33:36 2022] PUT /identity/v3/projects/f942ab31bafc428bb1fba94aee2e0c32/users/0a26028451b345968fe86774f44736dd/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:33:36.928366 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7038607-70ca-41b0-bab5-08ca2ce6d5b0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:36.943725 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-02f1d31c-5fff-4d21-8114-c2998cd52248 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:36.945853 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-02f1d31c-5fff-4d21-8114-c2998cd52248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:37.008244 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7038607-70ca-41b0-bab5-08ca2ce6d5b0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:37.014808 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b588a9-75f0-41a5-8498-44fcde7398c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:37.019015 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b588a9-75f0-41a5-8498-44fcde7398c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:37.019349 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b588a9-75f0-41a5-8498-44fcde7398c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:37.019918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b588a9-75f0-41a5-8498-44fcde7398c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:37.020191 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:33:37.020712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b588a9-75f0-41a5-8498-44fcde7398c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:37.023141 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b588a9-75f0-41a5-8498-44fcde7398c5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:37.035659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 795/1594] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:33:37.036606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b588a9-75f0-41a5-8498-44fcde7398c5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:37.041508 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 800/1595] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:37.042516 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1d31c-5fff-4d21-8114-c2998cd52248 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:37.052174 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b588a9-75f0-41a5-8498-44fcde7398c5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:37.053740 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1d31c-5fff-4d21-8114-c2998cd52248 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:37.058175 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865c6e0-32fc-43ec-a5b2-5f7f7a82b343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:37.058953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865c6e0-32fc-43ec-a5b2-5f7f7a82b343 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:37.059133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a856ac51-edbf-4f54-a280-9f7d05abb992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:37.059331 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865c6e0-32fc-43ec-a5b2-5f7f7a82b343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:37.065594 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865c6e0-32fc-43ec-a5b2-5f7f7a82b343 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:37.065738 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856ac51-edbf-4f54-a280-9f7d05abb992 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:37.065738 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856ac51-edbf-4f54-a280-9f7d05abb992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:37.065882 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5865c6e0-32fc-43ec-a5b2-5f7f7a82b343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:37.065955 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856ac51-edbf-4f54-a280-9f7d05abb992 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:37.065955 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a856ac51-edbf-4f54-a280-9f7d05abb992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:37.066127 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5865c6e0-32fc-43ec-a5b2-5f7f7a82b343 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:37.066513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a856ac51-edbf-4f54-a280-9f7d05abb992 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:37.075454 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 801/1596] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:33:37.077269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a856ac51-edbf-4f54-a280-9f7d05abb992 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:37.081976 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 796/1597] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:33:37.081976 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865c6e0-32fc-43ec-a5b2-5f7f7a82b343 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:37.088255 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a856ac51-edbf-4f54-a280-9f7d05abb992 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:37.093997 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33260a-4501-4df4-9de9-af1531ffc69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:37.094792 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33260a-4501-4df4-9de9-af1531ffc69c admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:37.095089 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33260a-4501-4df4-9de9-af1531ffc69c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:37.095371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33260a-4501-4df4-9de9-af1531ffc69c admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:37.096436 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33260a-4501-4df4-9de9-af1531ffc69c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:37.098167 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33260a-4501-4df4-9de9-af1531ffc69c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:37.106393 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865c6e0-32fc-43ec-a5b2-5f7f7a82b343 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:37.110190 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 802/1598] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:33:37.113956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcbc65-f601-482c-9e9e-803461a542c3 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:37.113956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcbc65-f601-482c-9e9e-803461a542c3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:37.113956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dcbc65-f601-482c-9e9e-803461a542c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:37.114286 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58311ff-8154-466b-8c5e-fdb3535a8d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:37.114286 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58311ff-8154-466b-8c5e-fdb3535a8d99 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:37.114286 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58311ff-8154-466b-8c5e-fdb3535a8d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:37.114286 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58311ff-8154-466b-8c5e-fdb3535a8d99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:37.115286 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58311ff-8154-466b-8c5e-fdb3535a8d99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:37.117189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58311ff-8154-466b-8c5e-fdb3535a8d99 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:37.128823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 797/1599] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:33:37.129805 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58311ff-8154-466b-8c5e-fdb3535a8d99 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:37.130215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a4dcbc65-f601-482c-9e9e-803461a542c3 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:37.131447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a4dcbc65-f601-482c-9e9e-803461a542c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:37.140521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 803/1600] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:37.141552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dcbc65-f601-482c-9e9e-803461a542c3 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:37.141880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58311ff-8154-466b-8c5e-fdb3535a8d99 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:37.151388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf7ab25-f38b-4756-8d3b-e6b5ebbe94d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:37.152674 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7ab25-f38b-4756-8d3b-e6b5ebbe94d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:37.152961 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7ab25-f38b-4756-8d3b-e6b5ebbe94d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:37.153234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf7ab25-f38b-4756-8d3b-e6b5ebbe94d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:37.153976 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf7ab25-f38b-4756-8d3b-e6b5ebbe94d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:37.155712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf7ab25-f38b-4756-8d3b-e6b5ebbe94d1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:37.166418 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 798/1601] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:33:37.166418 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf7ab25-f38b-4756-8d3b-e6b5ebbe94d1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:37.174431 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:33:37.178775 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf7ab25-f38b-4756-8d3b-e6b5ebbe94d1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:37.183940 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c00752-ffb0-4c16-878a-fc1b49e622a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:37.184629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c00752-ffb0-4c16-878a-fc1b49e622a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:37.184919 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c00752-ffb0-4c16-878a-fc1b49e622a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:37.185204 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c00752-ffb0-4c16-878a-fc1b49e622a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:37.186057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c00752-ffb0-4c16-878a-fc1b49e622a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:37.187729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c00752-ffb0-4c16-878a-fc1b49e622a6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:37.202093 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 799/1602] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:33:37.202093 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c00752-ffb0-4c16-878a-fc1b49e622a6 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:37.210453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c00752-ffb0-4c16-878a-fc1b49e622a6 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:37.211312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dcbc65-f601-482c-9e9e-803461a542c3 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:37.216455 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3892037-ca8e-493e-94e4-ab0785608e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:37.217375 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3892037-ca8e-493e-94e4-ab0785608e35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:37.217642 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3892037-ca8e-493e-94e4-ab0785608e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:37.217927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3892037-ca8e-493e-94e4-ab0785608e35 admin admin] PATH_INFO: `/v3/projects/aff9cc8cb38d45b1abfbafd1b1c4d30c/users/f657742a762044b5a1ba8621069ca8b8/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:37.219492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8cb19-f89c-4e72-be29-2dddf1a2d18f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:37.220659 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8cb19-f89c-4e72-be29-2dddf1a2d18f service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:37.221120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8cb19-f89c-4e72-be29-2dddf1a2d18f service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:37.222340 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc8cb19-f89c-4e72-be29-2dddf1a2d18f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:37.233364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8cb19-f89c-4e72-be29-2dddf1a2d18f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:37.234841 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc8cb19-f89c-4e72-be29-2dddf1a2d18f service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:37.236031 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3892037-ca8e-493e-94e4-ab0785608e35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aff9cc8cb38d45b1abfbafd1b1c4d30c, user_id=f657742a762044b5a1ba8621069ca8b8, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:37.238818 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3892037-ca8e-493e-94e4-ab0785608e35 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:37.245935 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 804/1603] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:33:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:33:37.246811 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8cb19-f89c-4e72-be29-2dddf1a2d18f service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:37.257592 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc8cb19-f89c-4e72-be29-2dddf1a2d18f service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:37.264721 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0bd4a-8a51-47c4-bc91-9338c2b1bdb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:37.269825 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0bd4a-8a51-47c4-bc91-9338c2b1bdb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:37.269825 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0bd4a-8a51-47c4-bc91-9338c2b1bdb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:37.270318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f0bd4a-8a51-47c4-bc91-9338c2b1bdb8 admin admin] PATH_INFO: `/v3/projects/76d2ecde0a1f40428c88fb69fdacd032/users/dbedf4dce9614b6eb2123714fbb03b29/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:37.292148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f0bd4a-8a51-47c4-bc91-9338c2b1bdb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d2ecde0a1f40428c88fb69fdacd032, user_id=dbedf4dce9614b6eb2123714fbb03b29, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:37.296532 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: WARNING py.warnings [None req-e9f0bd4a-8a51-47c4-bc91-9338c2b1bdb8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 16:33:37.296532 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: warnings.warn(msg) Jun 30 16:33:37.296532 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: Jun 30 16:33:37.301973 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f0bd4a-8a51-47c4-bc91-9338c2b1bdb8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:37.321634 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 800/1604] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:33:37 2022] PUT /identity/v3/projects/aff9cc8cb38d45b1abfbafd1b1c4d30c/users/f657742a762044b5a1ba8621069ca8b8/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:33:37.325202 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3892037-ca8e-493e-94e4-ab0785608e35 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:37.395527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 805/1605] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:33:37 2022] PUT /identity/v3/projects/76d2ecde0a1f40428c88fb69fdacd032/users/dbedf4dce9614b6eb2123714fbb03b29/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:33:37.395527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0bd4a-8a51-47c4-bc91-9338c2b1bdb8 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:37.420125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3892037-ca8e-493e-94e4-ab0785608e35 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:37.492860 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f0bd4a-8a51-47c4-bc91-9338c2b1bdb8 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:37.497330 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c99253b-782a-40f6-9817-31bf3a8b27dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:37.497994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99253b-782a-40f6-9817-31bf3a8b27dc service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:37.498278 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99253b-782a-40f6-9817-31bf3a8b27dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:37.498558 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c99253b-782a-40f6-9817-31bf3a8b27dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:37.500944 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4805c8b-9fe2-4a10-9cc9-0313fd649c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:37.502036 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4805c8b-9fe2-4a10-9cc9-0313fd649c2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:37.502307 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4805c8b-9fe2-4a10-9cc9-0313fd649c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:37.502556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4805c8b-9fe2-4a10-9cc9-0313fd649c2e admin admin] PATH_INFO: `/v3/projects/80a9bf240ed0447faaf33bec8f19cb70/users/18f2856b1d3742a1aca89b4aeec0a3bd/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:37.509470 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c99253b-782a-40f6-9817-31bf3a8b27dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:37.516684 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c99253b-782a-40f6-9817-31bf3a8b27dc service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:37.521642 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4805c8b-9fe2-4a10-9cc9-0313fd649c2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=80a9bf240ed0447faaf33bec8f19cb70, user_id=18f2856b1d3742a1aca89b4aeec0a3bd, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:37.522583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 806/1606] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:33:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:33:37.523221 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c99253b-782a-40f6-9817-31bf3a8b27dc service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:37.525010 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4805c8b-9fe2-4a10-9cc9-0313fd649c2e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:37.533049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c99253b-782a-40f6-9817-31bf3a8b27dc service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:37.537931 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7aec2-c031-416d-9b94-8c2ef5c1c3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:37.538876 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7aec2-c031-416d-9b94-8c2ef5c1c3c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:37.539128 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7aec2-c031-416d-9b94-8c2ef5c1c3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:37.539406 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7aec2-c031-416d-9b94-8c2ef5c1c3c2 admin admin] PATH_INFO: `/v3/projects/cd8720b566d44431bd0848c7d6f4ca5c/users/dc4c24a998354c18a889c3a9f5f62a37/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:37.563947 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c7aec2-c031-416d-9b94-8c2ef5c1c3c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd8720b566d44431bd0848c7d6f4ca5c, user_id=dc4c24a998354c18a889c3a9f5f62a37, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:37.568174 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c7aec2-c031-416d-9b94-8c2ef5c1c3c2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:37.692577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 801/1607] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:33:37 2022] PUT /identity/v3/projects/80a9bf240ed0447faaf33bec8f19cb70/users/18f2856b1d3742a1aca89b4aeec0a3bd/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:33:37.692577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4805c8b-9fe2-4a10-9cc9-0313fd649c2e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:37.708131 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 807/1608] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:33:37 2022] PUT /identity/v3/projects/cd8720b566d44431bd0848c7d6f4ca5c/users/dc4c24a998354c18a889c3a9f5f62a37/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:33:37.709767 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721c40a-a870-4950-97a5-a0cd95e0e55f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:37.711637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721c40a-a870-4950-97a5-a0cd95e0e55f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:37.715780 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9721c40a-a870-4950-97a5-a0cd95e0e55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:37.763412 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9721c40a-a870-4950-97a5-a0cd95e0e55f None None] MFA Rules not processed for user `f657742a762044b5a1ba8621069ca8b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:37.765745 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9721c40a-a870-4950-97a5-a0cd95e0e55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:37.779983 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:33:37.791863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4805c8b-9fe2-4a10-9cc9-0313fd649c2e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:37.804123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a810a3-2546-46c6-9b33-e44919e88d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:37.805991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a810a3-2546-46c6-9b33-e44919e88d00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:37.806284 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a810a3-2546-46c6-9b33-e44919e88d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:37.806567 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a810a3-2546-46c6-9b33-e44919e88d00 admin admin] PATH_INFO: `/v3/projects/b66c389f8c5a47b2b97819e449c64ba0/users/7261ab46436e4546b8331aa8387cfa1c/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:37.812993 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:33:37.849447 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a810a3-2546-46c6-9b33-e44919e88d00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b66c389f8c5a47b2b97819e449c64ba0, user_id=7261ab46436e4546b8331aa8387cfa1c, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:37.852389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a810a3-2546-46c6-9b33-e44919e88d00 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:37.881008 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 808/1609] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:37.887827 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2de49-41f1-4db7-9fa0-273488749339 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:37.888203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2de49-41f1-4db7-9fa0-273488749339 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:37.888555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2de49-41f1-4db7-9fa0-273488749339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:37.930459 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9be2de49-41f1-4db7-9fa0-273488749339 None None] MFA Rules not processed for user `dbedf4dce9614b6eb2123714fbb03b29`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:37.932062 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9be2de49-41f1-4db7-9fa0-273488749339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:37.967641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 802/1610] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:33:37 2022] PUT /identity/v3/projects/b66c389f8c5a47b2b97819e449c64ba0/users/7261ab46436e4546b8331aa8387cfa1c/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:33:37.972560 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d60dbf-bf5e-4f0f-9a6b-031045be90e2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:37.972853 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d60dbf-bf5e-4f0f-9a6b-031045be90e2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:37.973348 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d60dbf-bf5e-4f0f-9a6b-031045be90e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:38.006442 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-49d60dbf-bf5e-4f0f-9a6b-031045be90e2 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:38.006442 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-49d60dbf-bf5e-4f0f-9a6b-031045be90e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:38.087794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 809/1611] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:38.088501 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 803/1612] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:38.088501 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4a536-efbc-463e-909f-3172af364698 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:38.088733 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e0cd7-ebb2-4ad5-b5da-ae5728406ca1 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:38.088829 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4a536-efbc-463e-909f-3172af364698 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:38.088938 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e0cd7-ebb2-4ad5-b5da-ae5728406ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:38.089029 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e4a536-efbc-463e-909f-3172af364698 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:38.089131 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0e0cd7-ebb2-4ad5-b5da-ae5728406ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:38.089234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 804/1613] 10.222.0.67 () {54 vars in 961 bytes} [Thu Jun 30 16:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:33:38.092600 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cad8d-4f68-48f1-a07b-c661e564eaf1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:38.092887 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cad8d-4f68-48f1-a07b-c661e564eaf1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:38.093219 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cad8d-4f68-48f1-a07b-c661e564eaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:38.117265 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e92cad8d-4f68-48f1-a07b-c661e564eaf1 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:38.122952 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e92cad8d-4f68-48f1-a07b-c661e564eaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:38.138458 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 805/1614] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:38.140179 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92cad8d-4f68-48f1-a07b-c661e564eaf1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:38.157090 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92cad8d-4f68-48f1-a07b-c661e564eaf1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:38.164753 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b12f6-5497-49e7-94e3-bc4297730bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:38.165498 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b12f6-5497-49e7-94e3-bc4297730bff admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:38.165955 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b12f6-5497-49e7-94e3-bc4297730bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:38.166238 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b12f6-5497-49e7-94e3-bc4297730bff admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:38.167508 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b12f6-5497-49e7-94e3-bc4297730bff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:38.170941 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7f0e0cd7-ebb2-4ad5-b5da-ae5728406ca1 None None] MFA Rules not processed for user `dc4c24a998354c18a889c3a9f5f62a37`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:38.171271 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4b12f6-5497-49e7-94e3-bc4297730bff admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:38.173272 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7f0e0cd7-ebb2-4ad5-b5da-ae5728406ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:38.288872 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 810/1615] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:38.294727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07904e5a-b74f-4e58-8435-53af1a6927e6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:38.295034 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07904e5a-b74f-4e58-8435-53af1a6927e6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:38.295371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07904e5a-b74f-4e58-8435-53af1a6927e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:38.318072 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-07904e5a-b74f-4e58-8435-53af1a6927e6 None None] MFA Rules not processed for user `71cca3708386459197abf6c67505bc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:38.319323 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-07904e5a-b74f-4e58-8435-53af1a6927e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:38.337765 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 806/1616] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:33:38 2022] POST /identity/v3/projects => generated 381 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:33:38.339246 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b12f6-5497-49e7-94e3-bc4297730bff admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:38.397916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:33:38.445752 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4b12f6-5497-49e7-94e3-bc4297730bff admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:38.456627 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714e0e9-dbaa-4875-a081-1a355a672eb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:38.458977 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714e0e9-dbaa-4875-a081-1a355a672eb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:38.460036 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714e0e9-dbaa-4875-a081-1a355a672eb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:38.460856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714e0e9-dbaa-4875-a081-1a355a672eb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:38.481132 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 811/1617] 10.222.0.67 () {58 vars in 1058 bytes} [Thu Jun 30 16:33:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:38.482186 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07904e5a-b74f-4e58-8435-53af1a6927e6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:38.613951 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07904e5a-b74f-4e58-8435-53af1a6927e6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:38.639719 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714e0e9-dbaa-4875-a081-1a355a672eb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:38.641068 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f92bb-34d4-449c-9301-2ec03ba88f2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:38.642332 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714e0e9-dbaa-4875-a081-1a355a672eb9 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:38.644165 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f92bb-34d4-449c-9301-2ec03ba88f2c service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:38.644677 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f92bb-34d4-449c-9301-2ec03ba88f2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:38.645133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8f92bb-34d4-449c-9301-2ec03ba88f2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:38.656190 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 807/1618] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:33:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:33:38.657461 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714e0e9-dbaa-4875-a081-1a355a672eb9 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:38.662476 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f92bb-34d4-449c-9301-2ec03ba88f2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:38.664425 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8f92bb-34d4-449c-9301-2ec03ba88f2c service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:38.665176 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:33:38.672350 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714e0e9-dbaa-4875-a081-1a355a672eb9 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:38.678117 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754c754-7fce-47c5-ac63-262dd3725d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:38.679369 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754c754-7fce-47c5-ac63-262dd3725d5f admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:38.679973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754c754-7fce-47c5-ac63-262dd3725d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:38.680472 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754c754-7fce-47c5-ac63-262dd3725d5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:38.681563 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5754c754-7fce-47c5-ac63-262dd3725d5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:38.685581 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5754c754-7fce-47c5-ac63-262dd3725d5f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:38.688950 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 812/1619] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:33:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:33:38.690188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f92bb-34d4-449c-9301-2ec03ba88f2c service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:38.714670 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8f92bb-34d4-449c-9301-2ec03ba88f2c service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:38.723881 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcaca3-d3b2-4df1-9a44-32571ca3bd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:38.725189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcaca3-d3b2-4df1-9a44-32571ca3bd1e admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:38.728397 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcaca3-d3b2-4df1-9a44-32571ca3bd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:38.728397 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcaca3-d3b2-4df1-9a44-32571ca3bd1e admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:38.728397 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcaca3-d3b2-4df1-9a44-32571ca3bd1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:38.732226 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcaca3-d3b2-4df1-9a44-32571ca3bd1e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:38.849175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 813/1620] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:33:38 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:33:38.851055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcaca3-d3b2-4df1-9a44-32571ca3bd1e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:38.854038 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 808/1621] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:33:38 2022] POST /identity/v3/projects => generated 371 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:33:38.856327 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754c754-7fce-47c5-ac63-262dd3725d5f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:38.988497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754c754-7fce-47c5-ac63-262dd3725d5f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:39.001009 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce6da3-5e23-46dd-b45f-4ffc2a5088e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:39.002509 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce6da3-5e23-46dd-b45f-4ffc2a5088e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:39.003316 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce6da3-5e23-46dd-b45f-4ffc2a5088e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:39.004083 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce6da3-5e23-46dd-b45f-4ffc2a5088e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:39.005665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ce6da3-5e23-46dd-b45f-4ffc2a5088e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:39.013134 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ce6da3-5e23-46dd-b45f-4ffc2a5088e3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:39.060392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcaca3-d3b2-4df1-9a44-32571ca3bd1e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:39.063195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 809/1622] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:33:39.064949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce6da3-5e23-46dd-b45f-4ffc2a5088e3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:39.085120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce6da3-5e23-46dd-b45f-4ffc2a5088e3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:39.094579 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b87d4-2731-4aab-9d36-e63045a4c8f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:39.096704 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b87d4-2731-4aab-9d36-e63045a4c8f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:39.097468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b87d4-2731-4aab-9d36-e63045a4c8f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:39.098554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056b87d4-2731-4aab-9d36-e63045a4c8f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:39.100228 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b87d4-2731-4aab-9d36-e63045a4c8f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:39.106631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056b87d4-2731-4aab-9d36-e63045a4c8f9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:39.188393 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:33:39.233664 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df9783-328b-4f5a-b50b-0322de5ed19d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:39.235168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df9783-328b-4f5a-b50b-0322de5ed19d service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:39.235918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df9783-328b-4f5a-b50b-0322de5ed19d service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:39.236765 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df9783-328b-4f5a-b50b-0322de5ed19d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:39.310901 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 810/1623] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:33:39 2022] POST /identity/v3/users => generated 292 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:33:39.313591 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b87d4-2731-4aab-9d36-e63045a4c8f9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:39.343836 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056b87d4-2731-4aab-9d36-e63045a4c8f9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:39.351320 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57110a-4360-4abd-a971-dc7930c2b74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:39.352917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57110a-4360-4abd-a971-dc7930c2b74c admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:39.353571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57110a-4360-4abd-a971-dc7930c2b74c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:39.354367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57110a-4360-4abd-a971-dc7930c2b74c admin admin] PATH_INFO: `/v3/projects/1fe4aac9b31d4507969d16d3536abad3/users/a8e3c0b5cd9d4bc7a3f135065156adbc/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:39.357415 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df9783-328b-4f5a-b50b-0322de5ed19d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:39.362944 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df9783-328b-4f5a-b50b-0322de5ed19d service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:39.379963 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 814/1624] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:33:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:33:39.381691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df9783-328b-4f5a-b50b-0322de5ed19d service nova] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:39.401233 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57110a-4360-4abd-a971-dc7930c2b74c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fe4aac9b31d4507969d16d3536abad3, user_id=a8e3c0b5cd9d4bc7a3f135065156adbc, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:39.401753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df9783-328b-4f5a-b50b-0322de5ed19d service nova] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:39.406667 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57110a-4360-4abd-a971-dc7930c2b74c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:39.409988 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cdf366-870d-434e-9b76-82a38d45f7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:39.411018 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cdf366-870d-434e-9b76-82a38d45f7c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:39.411502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cdf366-870d-434e-9b76-82a38d45f7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:39.411990 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cdf366-870d-434e-9b76-82a38d45f7c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:39.413452 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cdf366-870d-434e-9b76-82a38d45f7c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:39.417618 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cdf366-870d-434e-9b76-82a38d45f7c1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:39.439293 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 815/1625] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:33:39.440701 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cdf366-870d-434e-9b76-82a38d45f7c1 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:39.453079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cdf366-870d-434e-9b76-82a38d45f7c1 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:39.458606 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a0d91-60c4-4ea3-ac00-1fa57b756abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:39.459643 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a0d91-60c4-4ea3-ac00-1fa57b756abc admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:39.460048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a0d91-60c4-4ea3-ac00-1fa57b756abc admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:39.460294 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a0d91-60c4-4ea3-ac00-1fa57b756abc admin admin] PATH_INFO: `/v3/projects/36b7f62cfa2c42cf94244f6d4142d397/users/8a35c6456fdb4b4ba650194bdedf2b52/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:39.486090 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a0d91-60c4-4ea3-ac00-1fa57b756abc admin admin] RBAC: Authorizing `identity:create_grant(project_id=36b7f62cfa2c42cf94244f6d4142d397, user_id=8a35c6456fdb4b4ba650194bdedf2b52, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:39.491588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a0d91-60c4-4ea3-ac00-1fa57b756abc admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:39.513313 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 811/1626] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:33:39 2022] PUT /identity/v3/projects/1fe4aac9b31d4507969d16d3536abad3/users/a8e3c0b5cd9d4bc7a3f135065156adbc/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:33:39.518170 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57110a-4360-4abd-a971-dc7930c2b74c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:39.620193 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 816/1627] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:33:39 2022] PUT /identity/v3/projects/36b7f62cfa2c42cf94244f6d4142d397/users/8a35c6456fdb4b4ba650194bdedf2b52/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:33:39.625074 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a0d91-60c4-4ea3-ac00-1fa57b756abc admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:39.648269 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57110a-4360-4abd-a971-dc7930c2b74c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:39.759808 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:33:39.783976 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a0d91-60c4-4ea3-ac00-1fa57b756abc admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:39.796259 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c11aa10-6e6b-49e2-baf9-81ddb25211bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:39.797703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11aa10-6e6b-49e2-baf9-81ddb25211bf admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:39.798358 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11aa10-6e6b-49e2-baf9-81ddb25211bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:39.799080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11aa10-6e6b-49e2-baf9-81ddb25211bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:39.800191 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c11aa10-6e6b-49e2-baf9-81ddb25211bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:39.802928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c11aa10-6e6b-49e2-baf9-81ddb25211bf admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:39.812158 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962e20a-cf00-423f-9128-5818c72a1deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:39.815647 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962e20a-cf00-423f-9128-5818c72a1deb admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:39.816257 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962e20a-cf00-423f-9128-5818c72a1deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:39.816760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962e20a-cf00-423f-9128-5818c72a1deb admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:39.817718 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962e20a-cf00-423f-9128-5818c72a1deb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:39.822255 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 812/1628] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:33:39.824911 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962e20a-cf00-423f-9128-5818c72a1deb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:39.825147 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65bbca-bf12-4a4b-8b9a-97263d018686 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:39.825675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65bbca-bf12-4a4b-8b9a-97263d018686 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:39.826245 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65bbca-bf12-4a4b-8b9a-97263d018686 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:39.833785 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 813/1629] 10.222.0.67 () {54 vars in 912 bytes} [Thu Jun 30 16:33:39 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:33:39.834787 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65bbca-bf12-4a4b-8b9a-97263d018686 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:39.850981 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 817/1630] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:33:39.853736 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ee6b8-9f19-49b7-bbe6-2f87cf2daf7d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:39.853736 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ee6b8-9f19-49b7-bbe6-2f87cf2daf7d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:39.853999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ee6b8-9f19-49b7-bbe6-2f87cf2daf7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:39.854507 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65bbca-bf12-4a4b-8b9a-97263d018686 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:39.860142 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad11032-08d5-4dae-8302-2e4a65feacbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:39.861563 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad11032-08d5-4dae-8302-2e4a65feacbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:39.862235 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad11032-08d5-4dae-8302-2e4a65feacbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:39.863018 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad11032-08d5-4dae-8302-2e4a65feacbf admin admin] PATH_INFO: `/v3/projects/1fe4aac9b31d4507969d16d3536abad3/users/a8e3c0b5cd9d4bc7a3f135065156adbc/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:39.865550 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad11032-08d5-4dae-8302-2e4a65feacbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fe4aac9b31d4507969d16d3536abad3, user_id=a8e3c0b5cd9d4bc7a3f135065156adbc, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:39.869475 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad11032-08d5-4dae-8302-2e4a65feacbf admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:39.885132 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e18ee6b8-9f19-49b7-bbe6-2f87cf2daf7d None None] MFA Rules not processed for user `71cca3708386459197abf6c67505bc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:39.896437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e18ee6b8-9f19-49b7-bbe6-2f87cf2daf7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:39.897222 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 814/1631] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:33:39 2022] PUT /identity/v3/projects/1fe4aac9b31d4507969d16d3536abad3/users/a8e3c0b5cd9d4bc7a3f135065156adbc/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:33:39.898487 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad11032-08d5-4dae-8302-2e4a65feacbf admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:39.997917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 818/1632] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 30 16:33:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:40.092533 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc9aee-609c-4ed8-9ea1-31b82d7a0225 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:40.092533 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc9aee-609c-4ed8-9ea1-31b82d7a0225 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:40.092533 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc9aee-609c-4ed8-9ea1-31b82d7a0225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:40.094042 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad11032-08d5-4dae-8302-2e4a65feacbf admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:40.094042 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963f0db-4401-4e17-8eff-c7d844210b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:40.094042 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963f0db-4401-4e17-8eff-c7d844210b1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:40.094042 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963f0db-4401-4e17-8eff-c7d844210b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:40.094543 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963f0db-4401-4e17-8eff-c7d844210b1e admin admin] PATH_INFO: `/v3/projects/36b7f62cfa2c42cf94244f6d4142d397/users/8a35c6456fdb4b4ba650194bdedf2b52/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:40.094543 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8963f0db-4401-4e17-8eff-c7d844210b1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=36b7f62cfa2c42cf94244f6d4142d397, user_id=8a35c6456fdb4b4ba650194bdedf2b52, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:40.094543 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8963f0db-4401-4e17-8eff-c7d844210b1e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:40.113318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f3dc9aee-609c-4ed8-9ea1-31b82d7a0225 None None] MFA Rules not processed for user `a8e3c0b5cd9d4bc7a3f135065156adbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:40.118157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 815/1633] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:33:39 2022] PUT /identity/v3/projects/36b7f62cfa2c42cf94244f6d4142d397/users/8a35c6456fdb4b4ba650194bdedf2b52/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:33:40.119691 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963f0db-4401-4e17-8eff-c7d844210b1e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:40.121121 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f3dc9aee-609c-4ed8-9ea1-31b82d7a0225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:40.279403 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963f0db-4401-4e17-8eff-c7d844210b1e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:40.294880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8fcd29-8945-4976-8a30-cf3c8752fcf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:40.297139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fcd29-8945-4976-8a30-cf3c8752fcf3 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:40.298666 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fcd29-8945-4976-8a30-cf3c8752fcf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:40.299118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fcd29-8945-4976-8a30-cf3c8752fcf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:40.350300 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 819/1634] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:39 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:40.350300 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ad052-2fbc-4f9d-830a-b13655376fec None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:40.350300 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ad052-2fbc-4f9d-830a-b13655376fec None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:40.350300 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ad052-2fbc-4f9d-830a-b13655376fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:40.423292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d50ad052-2fbc-4f9d-830a-b13655376fec None None] MFA Rules not processed for user `8a35c6456fdb4b4ba650194bdedf2b52`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:40.424992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d50ad052-2fbc-4f9d-830a-b13655376fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:40.439577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8fcd29-8945-4976-8a30-cf3c8752fcf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:40.441586 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8fcd29-8945-4976-8a30-cf3c8752fcf3 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:40.462156 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 816/1635] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:33:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:33:40.467907 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8fcd29-8945-4976-8a30-cf3c8752fcf3 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:40.486162 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8fcd29-8945-4976-8a30-cf3c8752fcf3 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:40.492843 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b28c4-0a4e-4710-b56b-a9a4b8bf68c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:40.493467 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b28c4-0a4e-4710-b56b-a9a4b8bf68c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:40.493753 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b28c4-0a4e-4710-b56b-a9a4b8bf68c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:40.494031 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b28c4-0a4e-4710-b56b-a9a4b8bf68c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:40.505397 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b28c4-0a4e-4710-b56b-a9a4b8bf68c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:40.507226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7b28c4-0a4e-4710-b56b-a9a4b8bf68c1 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:40.524729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 817/1636] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:33:40 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:33:40.602008 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 820/1637] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:40 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:40.740243 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b28c4-0a4e-4710-b56b-a9a4b8bf68c1 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:40.744011 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:33:40.753006 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b28c4-0a4e-4710-b56b-a9a4b8bf68c1 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:40.759505 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696dbc7-1ab9-4bb3-86d9-50f003ad85c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:40.760491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696dbc7-1ab9-4bb3-86d9-50f003ad85c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:40.760868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696dbc7-1ab9-4bb3-86d9-50f003ad85c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:40.762283 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0696dbc7-1ab9-4bb3-86d9-50f003ad85c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:40.777659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0696dbc7-1ab9-4bb3-86d9-50f003ad85c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:40.779922 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0696dbc7-1ab9-4bb3-86d9-50f003ad85c0 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:40.825011 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 818/1638] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:33:40 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:33:41.189539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:33:41.336189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ad052-2fbc-4f9d-830a-b13655376fec None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:41.348862 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ad052-2fbc-4f9d-830a-b13655376fec None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:41.359649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd55e54-2ea8-45bc-8bac-9196d00961af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:41.361119 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd55e54-2ea8-45bc-8bac-9196d00961af service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:41.368832 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd55e54-2ea8-45bc-8bac-9196d00961af service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:41.368832 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd55e54-2ea8-45bc-8bac-9196d00961af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:41.523552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd55e54-2ea8-45bc-8bac-9196d00961af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:41.525174 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd55e54-2ea8-45bc-8bac-9196d00961af service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:41.549852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 821/1639] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:33:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:33:42.119932 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696dbc7-1ab9-4bb3-86d9-50f003ad85c0 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:42.129711 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0696dbc7-1ab9-4bb3-86d9-50f003ad85c0 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:42.144647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fdf65a-5ec6-46c8-940f-8dd6d23bef00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:42.146074 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdf65a-5ec6-46c8-940f-8dd6d23bef00 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:42.146392 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdf65a-5ec6-46c8-940f-8dd6d23bef00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:42.146629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fdf65a-5ec6-46c8-940f-8dd6d23bef00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:42.410816 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:33:42.514802 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fdf65a-5ec6-46c8-940f-8dd6d23bef00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:42.516315 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fdf65a-5ec6-46c8-940f-8dd6d23bef00 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:42.534847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 819/1640] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:33:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:33:43.860141 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ba14c-fbc3-4193-b5ff-4b6fd3394d24 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:43.860712 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ba14c-fbc3-4193-b5ff-4b6fd3394d24 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:43.860712 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ba14c-fbc3-4193-b5ff-4b6fd3394d24 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:43.865011 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 822/1641] 10.222.0.67 () {54 vars in 961 bytes} [Thu Jun 30 16:33:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:33:43.889604 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f638780-b7f2-4fc0-a1ae-f5d703cb060f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:43.896127 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f638780-b7f2-4fc0-a1ae-f5d703cb060f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:43.896627 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f638780-b7f2-4fc0-a1ae-f5d703cb060f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:43.937447 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5f638780-b7f2-4fc0-a1ae-f5d703cb060f None None] MFA Rules not processed for user `71cca3708386459197abf6c67505bc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:43.938943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5f638780-b7f2-4fc0-a1ae-f5d703cb060f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:43.952535 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 820/1642] 10.222.0.67 () {58 vars in 1058 bytes} [Thu Jun 30 16:33:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:43.962467 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18da72c-69cc-489c-bce5-be7b3437f43f None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:43.962763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18da72c-69cc-489c-bce5-be7b3437f43f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:43.963080 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18da72c-69cc-489c-bce5-be7b3437f43f None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:43.965800 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 823/1643] 10.222.0.67 () {54 vars in 912 bytes} [Thu Jun 30 16:33:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:33:43.975197 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0524b0-ef0b-407b-b1c5-bd394ba988e9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:43.975909 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0524b0-ef0b-407b-b1c5-bd394ba988e9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:43.977586 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0524b0-ef0b-407b-b1c5-bd394ba988e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:44.032004 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0d0524b0-ef0b-407b-b1c5-bd394ba988e9 None None] MFA Rules not processed for user `71cca3708386459197abf6c67505bc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:44.033876 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0d0524b0-ef0b-407b-b1c5-bd394ba988e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:44.068000 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 821/1644] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 30 16:33:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:44.078688 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18da72c-69cc-489c-bce5-be7b3437f43f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:44.110986 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18da72c-69cc-489c-bce5-be7b3437f43f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:44.112646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407c8f5-81c9-40c6-a561-6297fd4d872c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:44.113515 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407c8f5-81c9-40c6-a561-6297fd4d872c service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:44.113990 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407c8f5-81c9-40c6-a561-6297fd4d872c service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:44.114411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407c8f5-81c9-40c6-a561-6297fd4d872c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:44.141081 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6407c8f5-81c9-40c6-a561-6297fd4d872c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:44.143763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6407c8f5-81c9-40c6-a561-6297fd4d872c service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:44.178260 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 824/1645] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:33:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:33:50.008198 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ee241-3803-4428-8b33-b8b833184fdf None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:50.008746 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ee241-3803-4428-8b33-b8b833184fdf None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:50.009959 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ee241-3803-4428-8b33-b8b833184fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:50.072243 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7b0ee241-3803-4428-8b33-b8b833184fdf None None] MFA Rules not processed for user `5703438f487a45abbc5b68b77519534c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:50.077283 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7b0ee241-3803-4428-8b33-b8b833184fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:50.290947 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 822/1646] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:50.293271 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3dbe9c-668f-4de8-87aa-556fde3f9af2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:50.293783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3dbe9c-668f-4de8-87aa-556fde3f9af2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:50.294250 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3dbe9c-668f-4de8-87aa-556fde3f9af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:50.371464 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ee3dbe9c-668f-4de8-87aa-556fde3f9af2 None None] MFA Rules not processed for user `5703438f487a45abbc5b68b77519534c`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:50.383758 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ee3dbe9c-668f-4de8-87aa-556fde3f9af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:50.397862 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 825/1647] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:50.410014 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ee241-3803-4428-8b33-b8b833184fdf None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:50.433857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ee241-3803-4428-8b33-b8b833184fdf None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:50.458604 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb2c06-57bc-436b-84dd-6e0374e1ce09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:50.459534 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb2c06-57bc-436b-84dd-6e0374e1ce09 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:50.459994 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb2c06-57bc-436b-84dd-6e0374e1ce09 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:50.460962 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb2c06-57bc-436b-84dd-6e0374e1ce09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:50.484183 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb2c06-57bc-436b-84dd-6e0374e1ce09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:50.493701 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb2c06-57bc-436b-84dd-6e0374e1ce09 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:50.524723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 823/1648] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:33:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:33:51.981904 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6316444-7a75-458a-97ef-3c41d1681dfb None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:51.982439 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6316444-7a75-458a-97ef-3c41d1681dfb None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:51.985109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6316444-7a75-458a-97ef-3c41d1681dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:52.033927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a0d3f-78c3-4c4f-be72-19b2628cb4d5 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:52.034761 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a0d3f-78c3-4c4f-be72-19b2628cb4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:52.034761 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849a0d3f-78c3-4c4f-be72-19b2628cb4d5 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:52.036204 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 824/1649] 10.222.0.67 () {54 vars in 912 bytes} [Thu Jun 30 16:33:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:33:52.049998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de421daa-01e3-4c8f-8347-bc2311d50690 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:52.050286 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de421daa-01e3-4c8f-8347-bc2311d50690 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:52.050575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de421daa-01e3-4c8f-8347-bc2311d50690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:52.072665 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c6316444-7a75-458a-97ef-3c41d1681dfb None None] MFA Rules not processed for user `2de23770f4374decb95ff94f3e7f793b`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:52.074951 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c6316444-7a75-458a-97ef-3c41d1681dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:52.143963 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-de421daa-01e3-4c8f-8347-bc2311d50690 None None] MFA Rules not processed for user `9b508dd8b7244d5f979e1ce7787d2c67`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:52.153013 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-de421daa-01e3-4c8f-8347-bc2311d50690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:52.223972 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:33:52.305510 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 826/1650] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:52.307245 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51201226-31fa-4a45-bfa9-25a915d24a6e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:52.308204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51201226-31fa-4a45-bfa9-25a915d24a6e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:52.308624 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51201226-31fa-4a45-bfa9-25a915d24a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:52.328597 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-51201226-31fa-4a45-bfa9-25a915d24a6e None None] MFA Rules not processed for user `2de23770f4374decb95ff94f3e7f793b`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:52.329580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 825/1651] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 30 16:33:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:52.335706 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-51201226-31fa-4a45-bfa9-25a915d24a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:52.335969 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de421daa-01e3-4c8f-8347-bc2311d50690 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:52.348964 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 827/1652] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:52.355381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de421daa-01e3-4c8f-8347-bc2311d50690 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:52.364217 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51201226-31fa-4a45-bfa9-25a915d24a6e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:52.367121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad40772-b407-412b-b260-2aa836c85780 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:52.367845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad40772-b407-412b-b260-2aa836c85780 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:52.368126 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad40772-b407-412b-b260-2aa836c85780 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:52.368404 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad40772-b407-412b-b260-2aa836c85780 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:52.379911 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51201226-31fa-4a45-bfa9-25a915d24a6e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:52.388482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8943d-b158-4c43-a0ed-2160a3ca24a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:52.389232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8943d-b158-4c43-a0ed-2160a3ca24a6 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:52.389591 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8943d-b158-4c43-a0ed-2160a3ca24a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:52.389948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8943d-b158-4c43-a0ed-2160a3ca24a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:52.390855 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad40772-b407-412b-b260-2aa836c85780 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:52.392607 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad40772-b407-412b-b260-2aa836c85780 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:52.408003 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed8943d-b158-4c43-a0ed-2160a3ca24a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:52.413035 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 826/1653] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:33:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:33:52.418639 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed8943d-b158-4c43-a0ed-2160a3ca24a6 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:52.459378 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 828/1654] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:33:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:33:53.153072 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2bb69-7d71-434b-8539-945be0f5e385 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:53.153537 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2bb69-7d71-434b-8539-945be0f5e385 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:53.153616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2bb69-7d71-434b-8539-945be0f5e385 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:53.155578 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 827/1655] 10.222.0.67 () {54 vars in 912 bytes} [Thu Jun 30 16:33:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:33:53.169570 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a7487-5b28-4875-8d2f-4ed536610c6c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:53.169570 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a7487-5b28-4875-8d2f-4ed536610c6c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:53.170212 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a7487-5b28-4875-8d2f-4ed536610c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:53.207265 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-bc9a7487-5b28-4875-8d2f-4ed536610c6c None None] MFA Rules not processed for user `9b508dd8b7244d5f979e1ce7787d2c67`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:53.208893 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bc9a7487-5b28-4875-8d2f-4ed536610c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:53.239368 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 829/1656] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 30 16:33:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:53.241230 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2bb69-7d71-434b-8539-945be0f5e385 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:53.263210 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2bb69-7d71-434b-8539-945be0f5e385 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:53.267664 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117db753-fd50-4c91-b25a-900220e9733c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:53.268429 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117db753-fd50-4c91-b25a-900220e9733c service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:53.268787 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117db753-fd50-4c91-b25a-900220e9733c service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:53.269160 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117db753-fd50-4c91-b25a-900220e9733c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:53.312611 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117db753-fd50-4c91-b25a-900220e9733c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:53.314821 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117db753-fd50-4c91-b25a-900220e9733c service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:53.379999 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 828/1657] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:33:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:33:53.504240 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f5f75-8922-4fd9-9c00-31498eeb442b None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:53.504619 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f5f75-8922-4fd9-9c00-31498eeb442b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:53.504980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5f5f75-8922-4fd9-9c00-31498eeb442b None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:53.506776 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 830/1658] 10.222.0.67 () {54 vars in 960 bytes} [Thu Jun 30 16:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:33:53.522991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1d025-a942-44ba-a86a-563cecd3a2fd None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:53.523337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1d025-a942-44ba-a86a-563cecd3a2fd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:53.523654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1d025-a942-44ba-a86a-563cecd3a2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:53.584621 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-69a1d025-a942-44ba-a86a-563cecd3a2fd None None] MFA Rules not processed for user `a932a9390fdd47f3adcf053a60250944`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:53.586797 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-69a1d025-a942-44ba-a86a-563cecd3a2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:53.710528 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 829/1659] 10.222.0.67 () {58 vars in 1057 bytes} [Thu Jun 30 16:33:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:53.720256 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f5f75-8922-4fd9-9c00-31498eeb442b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:53.740602 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5f5f75-8922-4fd9-9c00-31498eeb442b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:53.746690 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acaf829-345f-4b28-80c8-d64643116907 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a932a9390fdd47f3adcf053a60250944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:53.747370 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acaf829-345f-4b28-80c8-d64643116907 service placement] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:53.747693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acaf829-345f-4b28-80c8-d64643116907 service placement] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:53.747971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acaf829-345f-4b28-80c8-d64643116907 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:53.749912 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4acaf829-345f-4b28-80c8-d64643116907 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:53.764045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acaf829-345f-4b28-80c8-d64643116907 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:53.765679 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acaf829-345f-4b28-80c8-d64643116907 service placement] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:53.803821 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 831/1660] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:33:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:33:54.596339 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102f679-053c-42f8-99d6-7df00664b902 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:54.597821 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102f679-053c-42f8-99d6-7df00664b902 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:54.598600 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102f679-053c-42f8-99d6-7df00664b902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:54.709937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0102f679-053c-42f8-99d6-7df00664b902 None None] MFA Rules not processed for user `0a26028451b345968fe86774f44736dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:54.715524 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0102f679-053c-42f8-99d6-7df00664b902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:54.959919 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 830/1661] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:55.061242 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e0673-df4d-45ec-8168-0a934268610c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:55.061989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e0673-df4d-45ec-8168-0a934268610c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:55.061989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e0673-df4d-45ec-8168-0a934268610c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:55.143593 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3b7e0673-df4d-45ec-8168-0a934268610c None None] MFA Rules not processed for user `2de23770f4374decb95ff94f3e7f793b`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:55.153816 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3b7e0673-df4d-45ec-8168-0a934268610c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:55.186733 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 832/1662] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:55.194751 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102f679-053c-42f8-99d6-7df00664b902 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:55.231050 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102f679-053c-42f8-99d6-7df00664b902 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:55.267644 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fd738-29e7-46c0-a265-89f3c0af1a23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:55.268884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fd738-29e7-46c0-a265-89f3c0af1a23 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:55.268884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fd738-29e7-46c0-a265-89f3c0af1a23 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:55.268884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fd738-29e7-46c0-a265-89f3c0af1a23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:55.316011 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4fd738-29e7-46c0-a265-89f3c0af1a23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:55.323981 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4fd738-29e7-46c0-a265-89f3c0af1a23 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:55.386328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 831/1663] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:33:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:33:58.469432 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e0673-df4d-45ec-8168-0a934268610c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:58.758510 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e0673-df4d-45ec-8168-0a934268610c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:33:58.777937 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a5f90-061b-4631-aca0-da1d89dcea17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:33:58.779040 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a5f90-061b-4631-aca0-da1d89dcea17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:58.779328 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a5f90-061b-4631-aca0-da1d89dcea17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:58.779634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a5f90-061b-4631-aca0-da1d89dcea17 admin admin] PATH_INFO: `/v3/users/f657742a762044b5a1ba8621069ca8b8` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:58.783756 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a5f90-061b-4631-aca0-da1d89dcea17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f657742a762044b5a1ba8621069ca8b8)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:33:58.787738 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: WARNING py.warnings [None req-fc6a5f90-061b-4631-aca0-da1d89dcea17 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 16:33:58.787738 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: warnings.warn(msg) Jun 30 16:33:58.787738 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: Jun 30 16:33:58.791305 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a5f90-061b-4631-aca0-da1d89dcea17 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:33:58.814896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:33:59.244380 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fc6a5f90-061b-4631-aca0-da1d89dcea17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f657742a762044b5a1ba8621069ca8b8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:33:59.246557 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fc6a5f90-061b-4631-aca0-da1d89dcea17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f657742a762044b5a1ba8621069ca8b8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:33:59.283042 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fc6a5f90-061b-4631-aca0-da1d89dcea17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f657742a762044b5a1ba8621069ca8b8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:33:59.365689 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fc6a5f90-061b-4631-aca0-da1d89dcea17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f657742a762044b5a1ba8621069ca8b8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:33:59.367166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-fc6a5f90-061b-4631-aca0-da1d89dcea17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f657742a762044b5a1ba8621069ca8b8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:33:59.482879 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 833/1664] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:33:58 2022] DELETE /identity/v3/users/f657742a762044b5a1ba8621069ca8b8 => generated 0 bytes in 1012 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:33:59.488041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d725299-7fc0-446d-90f4-f6351ea34b16 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:33:59.489533 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d725299-7fc0-446d-90f4-f6351ea34b16 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:33:59.490305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d725299-7fc0-446d-90f4-f6351ea34b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:33:59.521698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8d725299-7fc0-446d-90f4-f6351ea34b16 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:33:59.528181 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8d725299-7fc0-446d-90f4-f6351ea34b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:33:59.592289 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:33:59.752808 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 832/1665] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:33:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:33:59.781636 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a5f90-061b-4631-aca0-da1d89dcea17 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:33:59.789345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fc6a5f90-061b-4631-aca0-da1d89dcea17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:00.084886 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a5f90-061b-4631-aca0-da1d89dcea17 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:00.095229 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf9383-e269-437e-b6c1-ef0fc6293c89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:00.096235 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf9383-e269-437e-b6c1-ef0fc6293c89 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:00.098479 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf9383-e269-437e-b6c1-ef0fc6293c89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:00.099121 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf9383-e269-437e-b6c1-ef0fc6293c89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:00.125321 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf9383-e269-437e-b6c1-ef0fc6293c89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:00.127420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf9383-e269-437e-b6c1-ef0fc6293c89 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:00.158612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 834/1666] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:33:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:34:00.393535 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf7645-de2c-4964-a194-8efa7b3d1770 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:00.394051 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf7645-de2c-4964-a194-8efa7b3d1770 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:00.394768 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf7645-de2c-4964-a194-8efa7b3d1770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:00.499439 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ccaf7645-de2c-4964-a194-8efa7b3d1770 None None] MFA Rules not processed for user `18f2856b1d3742a1aca89b4aeec0a3bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:00.501640 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ccaf7645-de2c-4964-a194-8efa7b3d1770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:00.677647 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f2bce-b768-4692-9828-36556119f14e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:00.678215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f2bce-b768-4692-9828-36556119f14e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:00.678302 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f2bce-b768-4692-9828-36556119f14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:00.715562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0f2f2bce-b768-4692-9828-36556119f14e None None] MFA Rules not processed for user `d49604bffa3d4e89a631eefa6d4e4f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:00.721052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0f2f2bce-b768-4692-9828-36556119f14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:00.859620 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 833/1667] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:00.874471 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 835/1668] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 30 16:34:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:34:00.921050 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf7645-de2c-4964-a194-8efa7b3d1770 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:00.922841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ccaf7645-de2c-4964-a194-8efa7b3d1770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:01.222755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf7645-de2c-4964-a194-8efa7b3d1770 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:01.230572 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c3f2b-5d83-4c00-97c5-3a9c8cb93cc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:01.232421 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c3f2b-5d83-4c00-97c5-3a9c8cb93cc8 service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:01.233055 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c3f2b-5d83-4c00-97c5-3a9c8cb93cc8 service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:01.233659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c3f2b-5d83-4c00-97c5-3a9c8cb93cc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:01.253474 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c3f2b-5d83-4c00-97c5-3a9c8cb93cc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:01.256437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c3f2b-5d83-4c00-97c5-3a9c8cb93cc8 service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:01.292757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 834/1669] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:34:01.516987 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f2bce-b768-4692-9828-36556119f14e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:01.518547 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0f2f2bce-b768-4692-9828-36556119f14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:01.597151 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:01.648554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f2bce-b768-4692-9828-36556119f14e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:01.655769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b87821-a07b-4867-a7c8-a2f6f1789257 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988b917eea4b48ffae32e23b8555a5c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:01.660066 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b87821-a07b-4867-a7c8-a2f6f1789257 service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:01.660066 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b87821-a07b-4867-a7c8-a2f6f1789257 service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:01.660066 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b87821-a07b-4867-a7c8-a2f6f1789257 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:01.672229 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b87821-a07b-4867-a7c8-a2f6f1789257 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:01.674542 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b87821-a07b-4867-a7c8-a2f6f1789257 service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:01.687493 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 836/1670] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 30 16:34:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 16:34:02.484034 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a4b73-9798-47cc-9108-7ea38e7fc6a1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:02.484728 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a4b73-9798-47cc-9108-7ea38e7fc6a1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:02.485086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a4b73-9798-47cc-9108-7ea38e7fc6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:02.613662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-dd7a4b73-9798-47cc-9108-7ea38e7fc6a1 None None] MFA Rules not processed for user `7261ab46436e4546b8331aa8387cfa1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:02.614786 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dd7a4b73-9798-47cc-9108-7ea38e7fc6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:02.654644 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:02.777241 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 835/1671] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:02.793310 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b87821-a07b-4867-a7c8-a2f6f1789257 service swift] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:02.795322 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c5b87821-a07b-4867-a7c8-a2f6f1789257 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:02.846676 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b87821-a07b-4867-a7c8-a2f6f1789257 service swift] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:02.875884 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12658940-29ca-4e2b-9501-771fe88a04c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:02.877298 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12658940-29ca-4e2b-9501-771fe88a04c9 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:02.878170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12658940-29ca-4e2b-9501-771fe88a04c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:02.878930 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12658940-29ca-4e2b-9501-771fe88a04c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:02.892596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12658940-29ca-4e2b-9501-771fe88a04c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:02.895344 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12658940-29ca-4e2b-9501-771fe88a04c9 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:02.949063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 837/1672] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:34:03.984198 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a4b73-9798-47cc-9108-7ea38e7fc6a1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:03.986438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dd7a4b73-9798-47cc-9108-7ea38e7fc6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:04.019346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12658940-29ca-4e2b-9501-771fe88a04c9 service nova] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:04.019346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-12658940-29ca-4e2b-9501-771fe88a04c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:04.045544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a4b73-9798-47cc-9108-7ea38e7fc6a1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:04.061623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12658940-29ca-4e2b-9501-771fe88a04c9 service nova] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:04.066407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884b944-4c97-48b9-b827-e940a3c69ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:04.072368 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884b944-4c97-48b9-b827-e940a3c69ceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:04.072860 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884b944-4c97-48b9-b827-e940a3c69ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:04.075740 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884b944-4c97-48b9-b827-e940a3c69ceb admin admin] PATH_INFO: `/v3/users/dbedf4dce9614b6eb2123714fbb03b29` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:04.077097 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db5763-34c3-4164-a128-caaf14af069f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:04.080382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9884b944-4c97-48b9-b827-e940a3c69ceb admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbedf4dce9614b6eb2123714fbb03b29)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:04.082312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db5763-34c3-4164-a128-caaf14af069f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:04.082604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db5763-34c3-4164-a128-caaf14af069f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:04.082892 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db5763-34c3-4164-a128-caaf14af069f admin admin] PATH_INFO: `/v3/projects/aff9cc8cb38d45b1abfbafd1b1c4d30c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:04.090672 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db5763-34c3-4164-a128-caaf14af069f admin admin] RBAC: Authorizing `identity:delete_project(project_id=aff9cc8cb38d45b1abfbafd1b1c4d30c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:04.091321 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9884b944-4c97-48b9-b827-e940a3c69ceb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:04.096882 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db5763-34c3-4164-a128-caaf14af069f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:04.506161 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9884b944-4c97-48b9-b827-e940a3c69ceb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbedf4dce9614b6eb2123714fbb03b29'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:04.508450 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-94db5763-34c3-4164-a128-caaf14af069f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aff9cc8cb38d45b1abfbafd1b1c4d30c', 'request_id': 'req-94db5763-34c3-4164-a128-caaf14af069f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:04.511434 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9884b944-4c97-48b9-b827-e940a3c69ceb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbedf4dce9614b6eb2123714fbb03b29'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:04.678316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-94db5763-34c3-4164-a128-caaf14af069f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aff9cc8cb38d45b1abfbafd1b1c4d30c', 'request_id': 'req-94db5763-34c3-4164-a128-caaf14af069f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:04.706817 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-94db5763-34c3-4164-a128-caaf14af069f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aff9cc8cb38d45b1abfbafd1b1c4d30c', 'request_id': 'req-94db5763-34c3-4164-a128-caaf14af069f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:04.795642 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9884b944-4c97-48b9-b827-e940a3c69ceb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbedf4dce9614b6eb2123714fbb03b29'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:04.813808 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-94db5763-34c3-4164-a128-caaf14af069f admin admin] The token cache is being invalidate because project aff9cc8cb38d45b1abfbafd1b1c4d30c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:34:04.814314 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-94db5763-34c3-4164-a128-caaf14af069f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:04.821948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-94db5763-34c3-4164-a128-caaf14af069f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:04.828631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9884b944-4c97-48b9-b827-e940a3c69ceb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbedf4dce9614b6eb2123714fbb03b29'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:04.841229 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9884b944-4c97-48b9-b827-e940a3c69ceb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbedf4dce9614b6eb2123714fbb03b29'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:04.849839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 838/1673] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:34:04 2022] DELETE /identity/v3/projects/aff9cc8cb38d45b1abfbafd1b1c4d30c => generated 0 bytes in 836 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:34:04.897349 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bafcf-cf7c-4f76-9c30-1cb122617b98 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:04.897637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bafcf-cf7c-4f76-9c30-1cb122617b98 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:04.901569 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bafcf-cf7c-4f76-9c30-1cb122617b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:04.968406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 836/1674] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:34:03 2022] DELETE /identity/v3/users/dbedf4dce9614b6eb2123714fbb03b29 => generated 0 bytes in 985 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:04.974164 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49d1a2-be46-44df-9395-0989fd57eb74 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:04.974449 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49d1a2-be46-44df-9395-0989fd57eb74 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:04.974730 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c49d1a2-be46-44df-9395-0989fd57eb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:05.020325 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8e7bafcf-cf7c-4f76-9c30-1cb122617b98 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:05.026521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8e7bafcf-cf7c-4f76-9c30-1cb122617b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:05.091055 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7c49d1a2-be46-44df-9395-0989fd57eb74 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:05.106290 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7c49d1a2-be46-44df-9395-0989fd57eb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:05.307538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 837/1675] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:05.318263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6513a-95e1-4399-92f5-bbf41155b11e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:05.318807 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6513a-95e1-4399-92f5-bbf41155b11e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:05.319299 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6513a-95e1-4399-92f5-bbf41155b11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:05.369566 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fcd6513a-95e1-4399-92f5-bbf41155b11e None None] MFA Rules not processed for user `9b508dd8b7244d5f979e1ce7787d2c67`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:05.382688 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 839/1676] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 533 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:05.387832 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bafcf-cf7c-4f76-9c30-1cb122617b98 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:05.389512 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fcd6513a-95e1-4399-92f5-bbf41155b11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:05.405819 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8e7bafcf-cf7c-4f76-9c30-1cb122617b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:05.550675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:05.564810 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:05.622150 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bafcf-cf7c-4f76-9c30-1cb122617b98 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:05.632270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c167106-b64e-42df-8a62-288899fed3d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:05.633171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c167106-b64e-42df-8a62-288899fed3d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:05.633612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c167106-b64e-42df-8a62-288899fed3d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:05.634104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c167106-b64e-42df-8a62-288899fed3d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:05.648817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 838/1677] 10.222.0.65 () {58 vars in 1015 bytes} [Thu Jun 30 16:34:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:05.652045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c167106-b64e-42df-8a62-288899fed3d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:05.652697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da1c36-a15c-4dbc-b3bc-6f47ec0a7ea2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:05.653175 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da1c36-a15c-4dbc-b3bc-6f47ec0a7ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:05.653645 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da1c36-a15c-4dbc-b3bc-6f47ec0a7ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:05.654971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c167106-b64e-42df-8a62-288899fed3d5 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:05.689003 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 840/1678] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:34:05.699412 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c167106-b64e-42df-8a62-288899fed3d5 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:05.724188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c167106-b64e-42df-8a62-288899fed3d5 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:05.732905 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a334f-fb0b-450d-ade4-b4280b2a7f8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:05.734033 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a334f-fb0b-450d-ade4-b4280b2a7f8f service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:05.739956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a334f-fb0b-450d-ade4-b4280b2a7f8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:05.741969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a334f-fb0b-450d-ade4-b4280b2a7f8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:05.760867 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-26da1c36-a15c-4dbc-b3bc-6f47ec0a7ea2 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:05.762124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-26da1c36-a15c-4dbc-b3bc-6f47ec0a7ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:05.770478 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a334f-fb0b-450d-ade4-b4280b2a7f8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:05.774817 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a334f-fb0b-450d-ade4-b4280b2a7f8f service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:05.812805 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 841/1679] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:34:05.814381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 839/1680] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:05.821277 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da1c36-a15c-4dbc-b3bc-6f47ec0a7ea2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:05.823506 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a334f-fb0b-450d-ade4-b4280b2a7f8f service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:05.826158 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fc9a334f-fb0b-450d-ade4-b4280b2a7f8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:05.857240 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da1c36-a15c-4dbc-b3bc-6f47ec0a7ea2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:05.873106 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a334f-fb0b-450d-ade4-b4280b2a7f8f service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:05.884294 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367edf68-a115-493a-b8d6-5ec0a0b9f14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:05.884714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d418f-0e7a-4271-9878-a8691bf344d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:05.885024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367edf68-a115-493a-b8d6-5ec0a0b9f14d admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:05.885313 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367edf68-a115-493a-b8d6-5ec0a0b9f14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:05.885595 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367edf68-a115-493a-b8d6-5ec0a0b9f14d admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:05.886315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d418f-0e7a-4271-9878-a8691bf344d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:05.888434 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d418f-0e7a-4271-9878-a8691bf344d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:05.889057 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d418f-0e7a-4271-9878-a8691bf344d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:05.894815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367edf68-a115-493a-b8d6-5ec0a0b9f14d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:34:05.897368 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367edf68-a115-493a-b8d6-5ec0a0b9f14d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:05.902471 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367edf68-a115-493a-b8d6-5ec0a0b9f14d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:05.911588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d418f-0e7a-4271-9878-a8691bf344d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:05.911588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d418f-0e7a-4271-9878-a8691bf344d5 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:05.938608 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 842/1681] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:34:05.959860 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 840/1682] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:34:05.965088 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d418f-0e7a-4271-9878-a8691bf344d5 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:05.990438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d418f-0e7a-4271-9878-a8691bf344d5 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:05.998552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db19f7-0860-4fec-a139-6df89216f694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:05.999650 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db19f7-0860-4fec-a139-6df89216f694 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:06.002064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db19f7-0860-4fec-a139-6df89216f694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:06.002064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db19f7-0860-4fec-a139-6df89216f694 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:06.002064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db19f7-0860-4fec-a139-6df89216f694 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:06.008131 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db19f7-0860-4fec-a139-6df89216f694 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:06.136277 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 843/1683] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:34:05 2022] POST /identity/v3/projects => generated 367 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:06.142414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367edf68-a115-493a-b8d6-5ec0a0b9f14d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:06.168041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367edf68-a115-493a-b8d6-5ec0a0b9f14d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:06.393193 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6276047-88cf-4a07-a58f-130f61f812c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:06.393193 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6276047-88cf-4a07-a58f-130f61f812c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:06.393193 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6276047-88cf-4a07-a58f-130f61f812c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:06.393193 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6276047-88cf-4a07-a58f-130f61f812c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:06.395211 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6276047-88cf-4a07-a58f-130f61f812c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:06.399497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6276047-88cf-4a07-a58f-130f61f812c5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:06.661730 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 841/1684] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:34:06 2022] POST /identity/v3/users => generated 284 bytes in 521 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:34:06.664027 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db19f7-0860-4fec-a139-6df89216f694 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:06.690349 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db19f7-0860-4fec-a139-6df89216f694 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:06.726606 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24129a-ab8b-4115-8ec2-36ea615a3509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:06.726606 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24129a-ab8b-4115-8ec2-36ea615a3509 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:06.726606 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24129a-ab8b-4115-8ec2-36ea615a3509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:06.729448 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c24129a-ab8b-4115-8ec2-36ea615a3509 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:06.731190 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24129a-ab8b-4115-8ec2-36ea615a3509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:06.743323 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c24129a-ab8b-4115-8ec2-36ea615a3509 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:06.783463 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 844/1685] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:06.786505 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6276047-88cf-4a07-a58f-130f61f812c5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:06.812355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6276047-88cf-4a07-a58f-130f61f812c5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:06.846544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8406da8d-c140-4b27-8f0a-32ba686ed43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:06.849770 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406da8d-c140-4b27-8f0a-32ba686ed43b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:06.850931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406da8d-c140-4b27-8f0a-32ba686ed43b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:06.852610 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406da8d-c140-4b27-8f0a-32ba686ed43b admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:06.862914 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8406da8d-c140-4b27-8f0a-32ba686ed43b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:06.865116 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8406da8d-c140-4b27-8f0a-32ba686ed43b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:06.941499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24129a-ab8b-4115-8ec2-36ea615a3509 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:06.945164 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 842/1686] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:06.954301 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c24129a-ab8b-4115-8ec2-36ea615a3509 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:06.968595 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae39393-569f-48d2-9674-7a692256ffe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:06.969891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae39393-569f-48d2-9674-7a692256ffe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:06.970490 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae39393-569f-48d2-9674-7a692256ffe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:06.970831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae39393-569f-48d2-9674-7a692256ffe2 admin admin] PATH_INFO: `/v3/projects/3af4529ca03d410bb181575d9e4cd1d6/users/823a876a766e4823abd4c2f7a0509b81/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:07.059617 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae39393-569f-48d2-9674-7a692256ffe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3af4529ca03d410bb181575d9e4cd1d6, user_id=823a876a766e4823abd4c2f7a0509b81, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:07.064662 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae39393-569f-48d2-9674-7a692256ffe2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:07.235429 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 845/1687] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:06 2022] PUT /identity/v3/projects/3af4529ca03d410bb181575d9e4cd1d6/users/823a876a766e4823abd4c2f7a0509b81/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:34:07.246692 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f706f-6ab8-4fb5-be43-4a5ad77614e0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:07.247204 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f706f-6ab8-4fb5-be43-4a5ad77614e0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:07.247303 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6f706f-6ab8-4fb5-be43-4a5ad77614e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:07.340352 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6c6f706f-6ab8-4fb5-be43-4a5ad77614e0 None None] MFA Rules not processed for user `823a876a766e4823abd4c2f7a0509b81`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:07.352041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6c6f706f-6ab8-4fb5-be43-4a5ad77614e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:07.612553 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 843/1688] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:07 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:07.614301 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae39393-569f-48d2-9674-7a692256ffe2 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:07.849391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae39393-569f-48d2-9674-7a692256ffe2 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:07.859176 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff64847-f21b-4f85-b84c-58d1fe3a91ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:07.860088 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff64847-f21b-4f85-b84c-58d1fe3a91ad admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:07.860856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff64847-f21b-4f85-b84c-58d1fe3a91ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:07.861208 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff64847-f21b-4f85-b84c-58d1fe3a91ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:07.861993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff64847-f21b-4f85-b84c-58d1fe3a91ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:07.864779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff64847-f21b-4f85-b84c-58d1fe3a91ad admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:08.060092 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 846/1689] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:34:07 2022] POST /identity/v3/projects => generated 369 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:34:08.070346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f706f-6ab8-4fb5-be43-4a5ad77614e0 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:08.131254 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6f706f-6ab8-4fb5-be43-4a5ad77614e0 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:08.294781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:08.384411 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576bc60e-b49f-4eb1-b8e3-af1929c6a9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:08.388723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576bc60e-b49f-4eb1-b8e3-af1929c6a9f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:08.389719 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576bc60e-b49f-4eb1-b8e3-af1929c6a9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:08.390259 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576bc60e-b49f-4eb1-b8e3-af1929c6a9f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:08.392718 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576bc60e-b49f-4eb1-b8e3-af1929c6a9f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:08.410380 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576bc60e-b49f-4eb1-b8e3-af1929c6a9f6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:08.573964 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 844/1690] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:34:08 2022] POST /identity/v3/users => generated 291 bytes in 505 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:08.579164 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff64847-f21b-4f85-b84c-58d1fe3a91ad admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:08.607635 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff64847-f21b-4f85-b84c-58d1fe3a91ad admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:08.631548 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b127b13-5f7a-4b4b-bba6-a5a645708259 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:08.631774 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b127b13-5f7a-4b4b-bba6-a5a645708259 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:08.631944 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b127b13-5f7a-4b4b-bba6-a5a645708259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:08.645754 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6d4cb-d706-448c-82cb-8de426590f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:08.652858 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6d4cb-d706-448c-82cb-8de426590f36 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:08.653309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6d4cb-d706-448c-82cb-8de426590f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:08.653754 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6d4cb-d706-448c-82cb-8de426590f36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:08.654651 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6d4cb-d706-448c-82cb-8de426590f36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:08.664088 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6d4cb-d706-448c-82cb-8de426590f36 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:08.667097 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5b127b13-5f7a-4b4b-bba6-a5a645708259 None None] MFA Rules not processed for user `0a26028451b345968fe86774f44736dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:08.668656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5b127b13-5f7a-4b4b-bba6-a5a645708259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:08.718247 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 847/1691] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:08.724570 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6d4cb-d706-448c-82cb-8de426590f36 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:08.752284 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6d4cb-d706-448c-82cb-8de426590f36 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:08.765259 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14986a71-7094-471d-923d-802cc2e13a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:08.766989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14986a71-7094-471d-923d-802cc2e13a4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:08.767448 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14986a71-7094-471d-923d-802cc2e13a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:08.767911 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14986a71-7094-471d-923d-802cc2e13a4a admin admin] PATH_INFO: `/v3/projects/b9a0cbda2dc945ea9c009036bb59c48a/users/4575435437bb48aba67e0aeca7e5b707/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:08.777198 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 845/1692] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:34:08.796254 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14986a71-7094-471d-923d-802cc2e13a4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9a0cbda2dc945ea9c009036bb59c48a, user_id=4575435437bb48aba67e0aeca7e5b707, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:08.812625 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14986a71-7094-471d-923d-802cc2e13a4a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:08.983562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 848/1693] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:08 2022] PUT /identity/v3/projects/b9a0cbda2dc945ea9c009036bb59c48a/users/4575435437bb48aba67e0aeca7e5b707/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:08.990903 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b127b13-5f7a-4b4b-bba6-a5a645708259 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:09.054370 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b127b13-5f7a-4b4b-bba6-a5a645708259 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:09.277954 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f112fe-69c6-426c-a868-4fca38497277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:09.279510 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f112fe-69c6-426c-a868-4fca38497277 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:09.279992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f112fe-69c6-426c-a868-4fca38497277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:09.280451 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f112fe-69c6-426c-a868-4fca38497277 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:09.282050 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f112fe-69c6-426c-a868-4fca38497277 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:09.284283 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f112fe-69c6-426c-a868-4fca38497277 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:09.307650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 846/1694] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:09.308293 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14986a71-7094-471d-923d-802cc2e13a4a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:09.315051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:09.335769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14986a71-7094-471d-923d-802cc2e13a4a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:09.361706 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24cee55-2f8c-4049-aaa9-f7af4a438e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:09.369211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cee55-2f8c-4049-aaa9-f7af4a438e1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:09.369529 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cee55-2f8c-4049-aaa9-f7af4a438e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:09.369818 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cee55-2f8c-4049-aaa9-f7af4a438e1f admin admin] PATH_INFO: `/v3/projects/b9a0cbda2dc945ea9c009036bb59c48a/users/4575435437bb48aba67e0aeca7e5b707/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:09.401392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24cee55-2f8c-4049-aaa9-f7af4a438e1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9a0cbda2dc945ea9c009036bb59c48a, user_id=4575435437bb48aba67e0aeca7e5b707, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:09.419439 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24cee55-2f8c-4049-aaa9-f7af4a438e1f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:09.449699 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f112fe-69c6-426c-a868-4fca38497277 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:09.453625 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-36f112fe-69c6-426c-a868-4fca38497277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:09.500575 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 849/1695] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:09 2022] PUT /identity/v3/projects/b9a0cbda2dc945ea9c009036bb59c48a/users/4575435437bb48aba67e0aeca7e5b707/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:09.508504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e0619-1007-4919-b842-c46b20b53f28 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:09.508866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e0619-1007-4919-b842-c46b20b53f28 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:09.509144 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0e0619-1007-4919-b842-c46b20b53f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:09.511123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f112fe-69c6-426c-a868-4fca38497277 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:09.609757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9a0e0619-1007-4919-b842-c46b20b53f28 None None] MFA Rules not processed for user `4575435437bb48aba67e0aeca7e5b707`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:09.614120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9a0e0619-1007-4919-b842-c46b20b53f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:09.742514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089df40-f16c-419d-a74d-5e1e689323c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:09.742638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089df40-f16c-419d-a74d-5e1e689323c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:09.742914 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089df40-f16c-419d-a74d-5e1e689323c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:09.743559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e089df40-f16c-419d-a74d-5e1e689323c8 admin admin] PATH_INFO: `/v3/projects/76d2ecde0a1f40428c88fb69fdacd032` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:09.750273 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e089df40-f16c-419d-a74d-5e1e689323c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76d2ecde0a1f40428c88fb69fdacd032)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:09.753369 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: WARNING py.warnings [None req-e089df40-f16c-419d-a74d-5e1e689323c8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 16:34:09.753369 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: warnings.warn(msg) Jun 30 16:34:09.753369 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: Jun 30 16:34:09.772910 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e089df40-f16c-419d-a74d-5e1e689323c8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:09.803949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:09.851366 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 850/1696] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:09 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:34:09.863993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e0619-1007-4919-b842-c46b20b53f28 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:09.866758 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9a0e0619-1007-4919-b842-c46b20b53f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:10.010046 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e089df40-f16c-419d-a74d-5e1e689323c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76d2ecde0a1f40428c88fb69fdacd032', 'request_id': 'req-e089df40-f16c-419d-a74d-5e1e689323c8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:10.051330 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0e0619-1007-4919-b842-c46b20b53f28 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:10.058757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e089df40-f16c-419d-a74d-5e1e689323c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76d2ecde0a1f40428c88fb69fdacd032', 'request_id': 'req-e089df40-f16c-419d-a74d-5e1e689323c8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:10.066065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e089df40-f16c-419d-a74d-5e1e689323c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76d2ecde0a1f40428c88fb69fdacd032', 'request_id': 'req-e089df40-f16c-419d-a74d-5e1e689323c8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:10.159372 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e089df40-f16c-419d-a74d-5e1e689323c8 admin admin] The token cache is being invalidate because project 76d2ecde0a1f40428c88fb69fdacd032 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:34:10.163620 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e089df40-f16c-419d-a74d-5e1e689323c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:10.165213 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e089df40-f16c-419d-a74d-5e1e689323c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:10.171832 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 847/1697] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:34:09 2022] DELETE /identity/v3/projects/76d2ecde0a1f40428c88fb69fdacd032 => generated 0 bytes in 725 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:10.176387 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089df40-f16c-419d-a74d-5e1e689323c8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:10.180942 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e089df40-f16c-419d-a74d-5e1e689323c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:10.283851 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d6bbd-d2aa-4ead-85e6-67a1d974b36f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:10.286589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d6bbd-d2aa-4ead-85e6-67a1d974b36f service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:10.288907 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d6bbd-d2aa-4ead-85e6-67a1d974b36f service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:10.289358 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d6bbd-d2aa-4ead-85e6-67a1d974b36f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:10.299003 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-092d6bbd-d2aa-4ead-85e6-67a1d974b36f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:10.429830 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e089df40-f16c-419d-a74d-5e1e689323c8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:10.441013 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb73cf-b825-467b-ae0c-c191b790e274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:10.441930 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb73cf-b825-467b-ae0c-c191b790e274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:10.442215 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb73cf-b825-467b-ae0c-c191b790e274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:10.445377 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb73cf-b825-467b-ae0c-c191b790e274 admin admin] PATH_INFO: `/v3/users/a8e3c0b5cd9d4bc7a3f135065156adbc` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:10.449599 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb73cf-b825-467b-ae0c-c191b790e274 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8e3c0b5cd9d4bc7a3f135065156adbc)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:10.454321 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb73cf-b825-467b-ae0c-c191b790e274 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:10.542367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d6bbd-d2aa-4ead-85e6-67a1d974b36f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:10.544008 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d6bbd-d2aa-4ead-85e6-67a1d974b36f service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:10.566306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 851/1698] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:09 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 703 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:34:10.752336 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4dfb73cf-b825-467b-ae0c-c191b790e274 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8e3c0b5cd9d4bc7a3f135065156adbc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:10.759427 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4dfb73cf-b825-467b-ae0c-c191b790e274 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8e3c0b5cd9d4bc7a3f135065156adbc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:10.903241 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4dfb73cf-b825-467b-ae0c-c191b790e274 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8e3c0b5cd9d4bc7a3f135065156adbc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:10.914847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4dfb73cf-b825-467b-ae0c-c191b790e274 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8e3c0b5cd9d4bc7a3f135065156adbc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:10.920009 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4dfb73cf-b825-467b-ae0c-c191b790e274 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8e3c0b5cd9d4bc7a3f135065156adbc'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:11.004085 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 848/1699] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:34:10 2022] DELETE /identity/v3/users/a8e3c0b5cd9d4bc7a3f135065156adbc => generated 0 bytes in 828 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:11.888706 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dadf6-74ae-46ca-9461-1a75a798bd65 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:11.889956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dadf6-74ae-46ca-9461-1a75a798bd65 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:11.890445 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53dadf6-74ae-46ca-9461-1a75a798bd65 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:11.897946 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 852/1700] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 30 16:34:11 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:34:11.913076 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2eff0-d2ed-4014-94e6-eeb8cfa5ae48 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:11.913571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2eff0-d2ed-4014-94e6-eeb8cfa5ae48 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:11.913782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2eff0-d2ed-4014-94e6-eeb8cfa5ae48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:11.983850 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7ee2eff0-d2ed-4014-94e6-eeb8cfa5ae48 None None] MFA Rules not processed for user `71cca3708386459197abf6c67505bc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:11.985251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7ee2eff0-d2ed-4014-94e6-eeb8cfa5ae48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:12.210451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:12.298861 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 849/1701] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 30 16:34:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:12.342118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:12.840494 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53dadf6-74ae-46ca-9461-1a75a798bd65 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:12.843162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e53dadf6-74ae-46ca-9461-1a75a798bd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:12.984738 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53dadf6-74ae-46ca-9461-1a75a798bd65 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:13.035148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796ce20-8c33-4520-81f6-7b7cfdc8ff7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:13.036245 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796ce20-8c33-4520-81f6-7b7cfdc8ff7a service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:13.036660 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796ce20-8c33-4520-81f6-7b7cfdc8ff7a service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:13.039350 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796ce20-8c33-4520-81f6-7b7cfdc8ff7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:13.058081 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5796ce20-8c33-4520-81f6-7b7cfdc8ff7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:13.060179 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5796ce20-8c33-4520-81f6-7b7cfdc8ff7a service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:13.077782 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 853/1702] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:34:13.189676 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6fcf0-cf5a-4e45-ba34-e146ac575164 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:13.190337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6fcf0-cf5a-4e45-ba34-e146ac575164 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:13.190337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6fcf0-cf5a-4e45-ba34-e146ac575164 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:13.192410 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 850/1703] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 30 16:34:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:34:13.324366 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3e993-5e2d-4784-b232-6e705197121d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:13.324697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3e993-5e2d-4784-b232-6e705197121d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:13.325012 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa3e993-5e2d-4784-b232-6e705197121d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:13.382341 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-bfa3e993-5e2d-4784-b232-6e705197121d None None] MFA Rules not processed for user `71cca3708386459197abf6c67505bc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:13.398326 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bfa3e993-5e2d-4784-b232-6e705197121d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:13.457322 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 854/1704] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 30 16:34:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:14.741555 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6fcf0-cf5a-4e45-ba34-e146ac575164 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:14.745987 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-89f6fcf0-cf5a-4e45-ba34-e146ac575164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:14.816431 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6fcf0-cf5a-4e45-ba34-e146ac575164 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:14.829655 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc39b3d-5824-41a2-9618-4576f0db389b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:14.830725 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc39b3d-5824-41a2-9618-4576f0db389b service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:14.831215 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc39b3d-5824-41a2-9618-4576f0db389b service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:14.837723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc39b3d-5824-41a2-9618-4576f0db389b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:14.850490 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc39b3d-5824-41a2-9618-4576f0db389b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:14.856368 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc39b3d-5824-41a2-9618-4576f0db389b service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:14.880204 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 851/1705] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:34:16.177317 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3e993-5e2d-4784-b232-6e705197121d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:16.181193 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bfa3e993-5e2d-4784-b232-6e705197121d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:16.540309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa3e993-5e2d-4784-b232-6e705197121d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:16.550765 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f35c2-d698-4ede-8445-89c3704ee913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:16.551722 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f35c2-d698-4ede-8445-89c3704ee913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:16.552006 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f35c2-d698-4ede-8445-89c3704ee913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:16.554193 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f35c2-d698-4ede-8445-89c3704ee913 admin admin] PATH_INFO: `/v3/projects/1fe4aac9b31d4507969d16d3536abad3` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:16.556958 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f35c2-d698-4ede-8445-89c3704ee913 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fe4aac9b31d4507969d16d3536abad3)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:16.565760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f35c2-d698-4ede-8445-89c3704ee913 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:16.648527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc39b3d-5824-41a2-9618-4576f0db389b service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:16.650868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dbc39b3d-5824-41a2-9618-4576f0db389b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:16.691170 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc39b3d-5824-41a2-9618-4576f0db389b service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:16.704381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b8b37-0d88-4a28-bf13-40801deb0793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:16.705671 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b8b37-0d88-4a28-bf13-40801deb0793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:16.706171 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b8b37-0d88-4a28-bf13-40801deb0793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:16.706660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b8b37-0d88-4a28-bf13-40801deb0793 admin admin] PATH_INFO: `/v3/users/823a876a766e4823abd4c2f7a0509b81` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:16.708343 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b8b37-0d88-4a28-bf13-40801deb0793 admin admin] RBAC: Authorizing `identity:delete_user(user_id=823a876a766e4823abd4c2f7a0509b81)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:16.710893 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b8b37-0d88-4a28-bf13-40801deb0793 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:16.791104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bd0f35c2-d698-4ede-8445-89c3704ee913 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fe4aac9b31d4507969d16d3536abad3', 'request_id': 'req-bd0f35c2-d698-4ede-8445-89c3704ee913'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:16.885105 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bd0f35c2-d698-4ede-8445-89c3704ee913 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fe4aac9b31d4507969d16d3536abad3', 'request_id': 'req-bd0f35c2-d698-4ede-8445-89c3704ee913'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:16.919491 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bd0f35c2-d698-4ede-8445-89c3704ee913 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fe4aac9b31d4507969d16d3536abad3', 'request_id': 'req-bd0f35c2-d698-4ede-8445-89c3704ee913'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:16.951148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bd0f35c2-d698-4ede-8445-89c3704ee913 admin admin] The token cache is being invalidate because project 1fe4aac9b31d4507969d16d3536abad3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:34:16.951980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bd0f35c2-d698-4ede-8445-89c3704ee913 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:16.953013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bd0f35c2-d698-4ede-8445-89c3704ee913 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:16.961177 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 855/1706] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:34:16 2022] DELETE /identity/v3/projects/1fe4aac9b31d4507969d16d3536abad3 => generated 0 bytes in 782 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:16.963741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e87e0-1c88-4d01-a403-e0ac8e2c7460 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:16.967210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e87e0-1c88-4d01-a403-e0ac8e2c7460 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:16.967862 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e87e0-1c88-4d01-a403-e0ac8e2c7460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:17.000732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-572e87e0-1c88-4d01-a403-e0ac8e2c7460 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:17.009491 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-572e87e0-1c88-4d01-a403-e0ac8e2c7460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:17.066816 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-080b8b37-0d88-4a28-bf13-40801deb0793 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '823a876a766e4823abd4c2f7a0509b81'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:17.070065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-080b8b37-0d88-4a28-bf13-40801deb0793 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '823a876a766e4823abd4c2f7a0509b81'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:17.193117 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-080b8b37-0d88-4a28-bf13-40801deb0793 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '823a876a766e4823abd4c2f7a0509b81'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:17.209706 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-080b8b37-0d88-4a28-bf13-40801deb0793 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '823a876a766e4823abd4c2f7a0509b81'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:17.209706 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-080b8b37-0d88-4a28-bf13-40801deb0793 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '823a876a766e4823abd4c2f7a0509b81'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:17.245526 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:17.326940 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 856/1708] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:17.335012 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 852/1708] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:34:16 2022] DELETE /identity/v3/users/823a876a766e4823abd4c2f7a0509b81 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:17.336408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57270b0d-db77-408f-8f90-03d5bb07c486 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:17.338956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03dc66-e6d4-456e-b5a8-67c651eb9c78 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:17.339358 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03dc66-e6d4-456e-b5a8-67c651eb9c78 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:17.339791 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03dc66-e6d4-456e-b5a8-67c651eb9c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:17.341389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57270b0d-db77-408f-8f90-03d5bb07c486 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:17.343335 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57270b0d-db77-408f-8f90-03d5bb07c486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:17.363331 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-4e03dc66-e6d4-456e-b5a8-67c651eb9c78 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:17.364824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4e03dc66-e6d4-456e-b5a8-67c651eb9c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:17.413958 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-57270b0d-db77-408f-8f90-03d5bb07c486 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:17.418895 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-57270b0d-db77-408f-8f90-03d5bb07c486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:17.604924 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 857/1709] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:17.607182 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03dc66-e6d4-456e-b5a8-67c651eb9c78 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:17.622169 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03dc66-e6d4-456e-b5a8-67c651eb9c78 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:17.629454 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6672f-77ac-4cf1-b1d9-6ce406b9ef02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:17.630296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6672f-77ac-4cf1-b1d9-6ce406b9ef02 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:17.630643 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6672f-77ac-4cf1-b1d9-6ce406b9ef02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:17.630991 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d6672f-77ac-4cf1-b1d9-6ce406b9ef02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:17.631923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6672f-77ac-4cf1-b1d9-6ce406b9ef02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:34:17.632277 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6672f-77ac-4cf1-b1d9-6ce406b9ef02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:17.634952 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d6672f-77ac-4cf1-b1d9-6ce406b9ef02 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:17.671640 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 858/1710] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:34:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:34:17.677269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6672f-77ac-4cf1-b1d9-6ce406b9ef02 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:17.691951 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d6672f-77ac-4cf1-b1d9-6ce406b9ef02 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:17.699618 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b6c97-ae48-464b-8ea4-bd9fad9a8a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:17.700617 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b6c97-ae48-464b-8ea4-bd9fad9a8a5c admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:17.701078 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b6c97-ae48-464b-8ea4-bd9fad9a8a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:17.701558 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b6c97-ae48-464b-8ea4-bd9fad9a8a5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:17.703689 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b6c97-ae48-464b-8ea4-bd9fad9a8a5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:17.706432 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b6c97-ae48-464b-8ea4-bd9fad9a8a5c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:17.754983 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 853/1711] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:17.819785 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57270b0d-db77-408f-8f90-03d5bb07c486 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:17.822177 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-57270b0d-db77-408f-8f90-03d5bb07c486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:17.835983 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:17.881880 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 859/1712] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:34:17 2022] POST /identity/v3/projects => generated 363 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:17.890333 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b6c97-ae48-464b-8ea4-bd9fad9a8a5c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:18.081492 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57270b0d-db77-408f-8f90-03d5bb07c486 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:18.128707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b6c97-ae48-464b-8ea4-bd9fad9a8a5c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:18.149232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02caf01-fd98-48bd-b0a1-034b9f6f7361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:18.150460 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02caf01-fd98-48bd-b0a1-034b9f6f7361 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:18.152530 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02caf01-fd98-48bd-b0a1-034b9f6f7361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:18.154559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02caf01-fd98-48bd-b0a1-034b9f6f7361 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:18.155575 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02caf01-fd98-48bd-b0a1-034b9f6f7361 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:18.160158 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02caf01-fd98-48bd-b0a1-034b9f6f7361 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:18.213963 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b30708-0db1-4342-819b-ffd961784b5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:18.214803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b30708-0db1-4342-819b-ffd961784b5f service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:18.215172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b30708-0db1-4342-819b-ffd961784b5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:18.215566 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b30708-0db1-4342-819b-ffd961784b5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:18.264485 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b30708-0db1-4342-819b-ffd961784b5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:18.269399 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b30708-0db1-4342-819b-ffd961784b5f service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:18.295061 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 854/1713] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:34:18.340380 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 860/1714] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:34:17 2022] POST /identity/v3/users => generated 282 bytes in 451 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:18.344601 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b30708-0db1-4342-819b-ffd961784b5f service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:18.359889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b30708-0db1-4342-819b-ffd961784b5f service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:18.372228 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded0917-dbe6-49df-9b68-ad58965932fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:18.373535 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded0917-dbe6-49df-9b68-ad58965932fd admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:18.374002 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded0917-dbe6-49df-9b68-ad58965932fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:18.374852 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ded0917-dbe6-49df-9b68-ad58965932fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:18.376245 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded0917-dbe6-49df-9b68-ad58965932fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:18.378321 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ded0917-dbe6-49df-9b68-ad58965932fd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:18.402149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 855/1715] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:34:18.407457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02caf01-fd98-48bd-b0a1-034b9f6f7361 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:18.423833 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02caf01-fd98-48bd-b0a1-034b9f6f7361 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:18.439384 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b25a4d-9a64-4e32-8d99-87f22db16f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:18.443807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b25a4d-9a64-4e32-8d99-87f22db16f96 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:18.444057 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b25a4d-9a64-4e32-8d99-87f22db16f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:18.444451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b25a4d-9a64-4e32-8d99-87f22db16f96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:18.445343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b25a4d-9a64-4e32-8d99-87f22db16f96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:18.452212 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b25a4d-9a64-4e32-8d99-87f22db16f96 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:18.480982 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 861/1716] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:18 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:18.491494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded0917-dbe6-49df-9b68-ad58965932fd admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:18.530291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ded0917-dbe6-49df-9b68-ad58965932fd admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:18.563780 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d00eb7-411b-4b4d-93de-0ad088884e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:18.564983 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d00eb7-411b-4b4d-93de-0ad088884e04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:18.565109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d00eb7-411b-4b4d-93de-0ad088884e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:18.565509 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d00eb7-411b-4b4d-93de-0ad088884e04 admin admin] PATH_INFO: `/v3/projects/bbc14b32ce7b47d18c84f545cb65b732/users/d8c444b440324b858ed8cd3f30924c36/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:18.629075 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d00eb7-411b-4b4d-93de-0ad088884e04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbc14b32ce7b47d18c84f545cb65b732, user_id=d8c444b440324b858ed8cd3f30924c36, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:18.640797 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d00eb7-411b-4b4d-93de-0ad088884e04 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:18.756225 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 856/1717] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:18 2022] PUT /identity/v3/projects/bbc14b32ce7b47d18c84f545cb65b732/users/d8c444b440324b858ed8cd3f30924c36/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:18.769226 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6fca6-a28c-4d3a-87b9-e306525da714 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:18.770191 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6fca6-a28c-4d3a-87b9-e306525da714 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:18.770674 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6fca6-a28c-4d3a-87b9-e306525da714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:18.867784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d2c6fca6-a28c-4d3a-87b9-e306525da714 None None] MFA Rules not processed for user `d8c444b440324b858ed8cd3f30924c36`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:18.872034 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d2c6fca6-a28c-4d3a-87b9-e306525da714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:19.043035 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 862/1718] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:18 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:19.047426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d00eb7-411b-4b4d-93de-0ad088884e04 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:19.189675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d00eb7-411b-4b4d-93de-0ad088884e04 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:19.207862 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b52ed-7146-42cb-bd91-4a7577dabe50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:19.207862 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b52ed-7146-42cb-bd91-4a7577dabe50 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:19.207862 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b52ed-7146-42cb-bd91-4a7577dabe50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:19.207862 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b52ed-7146-42cb-bd91-4a7577dabe50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:19.208375 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b52ed-7146-42cb-bd91-4a7577dabe50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:19.208375 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b52ed-7146-42cb-bd91-4a7577dabe50 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:19.401693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 857/1719] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:34:19 2022] POST /identity/v3/projects => generated 359 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:19.406755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6fca6-a28c-4d3a-87b9-e306525da714 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:19.445099 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1b01c-ebfd-4759-bb8e-040c71311e2c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:19.445476 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1b01c-ebfd-4759-bb8e-040c71311e2c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:19.445758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1b01c-ebfd-4759-bb8e-040c71311e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:19.452516 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6fca6-a28c-4d3a-87b9-e306525da714 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:19.502437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-40c1b01c-ebfd-4759-bb8e-040c71311e2c None None] MFA Rules not processed for user `18f2856b1d3742a1aca89b4aeec0a3bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:19.503814 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-40c1b01c-ebfd-4759-bb8e-040c71311e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:19.509806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:19.667041 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc4bfe-b570-432f-9830-7f601ef443ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:19.668659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:19.670986 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc4bfe-b570-432f-9830-7f601ef443ef admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:19.671282 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc4bfe-b570-432f-9830-7f601ef443ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:19.671593 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc4bfe-b570-432f-9830-7f601ef443ef admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:19.677096 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc4bfe-b570-432f-9830-7f601ef443ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:19.688612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc4bfe-b570-432f-9830-7f601ef443ef admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:19.709910 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 858/1720] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:34:20.081436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c1b01c-ebfd-4759-bb8e-040c71311e2c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:20.084260 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-40c1b01c-ebfd-4759-bb8e-040c71311e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:20.101893 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c1b01c-ebfd-4759-bb8e-040c71311e2c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:20.108292 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec6127-07c0-4569-81ab-bc56619f85ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:20.109437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec6127-07c0-4569-81ab-bc56619f85ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:20.109821 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec6127-07c0-4569-81ab-bc56619f85ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:20.110305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cec6127-07c0-4569-81ab-bc56619f85ab admin admin] PATH_INFO: `/v3/projects/3af4529ca03d410bb181575d9e4cd1d6` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:20.111967 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec6127-07c0-4569-81ab-bc56619f85ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=3af4529ca03d410bb181575d9e4cd1d6)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:20.115040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cec6127-07c0-4569-81ab-bc56619f85ab admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:20.215816 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 863/1721] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:34:19 2022] POST /identity/v3/users => generated 286 bytes in 809 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:20.219027 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc4bfe-b570-432f-9830-7f601ef443ef admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:20.243163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc4bfe-b570-432f-9830-7f601ef443ef admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:20.280340 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9cec6127-07c0-4569-81ab-bc56619f85ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3af4529ca03d410bb181575d9e4cd1d6', 'request_id': 'req-9cec6127-07c0-4569-81ab-bc56619f85ab'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:20.292813 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9cec6127-07c0-4569-81ab-bc56619f85ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3af4529ca03d410bb181575d9e4cd1d6', 'request_id': 'req-9cec6127-07c0-4569-81ab-bc56619f85ab'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:20.304333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9cec6127-07c0-4569-81ab-bc56619f85ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3af4529ca03d410bb181575d9e4cd1d6', 'request_id': 'req-9cec6127-07c0-4569-81ab-bc56619f85ab'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:20.341300 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c103484-1bcf-466d-a701-9434d60eacef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:20.342283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c103484-1bcf-466d-a701-9434d60eacef admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:20.342727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c103484-1bcf-466d-a701-9434d60eacef admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:20.343173 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c103484-1bcf-466d-a701-9434d60eacef admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:20.344311 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c103484-1bcf-466d-a701-9434d60eacef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:20.347972 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c103484-1bcf-466d-a701-9434d60eacef admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:20.361379 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 864/1722] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:20.365557 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a18ba-5ce3-4b8d-b3ea-7af8a3344591 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:20.365557 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a18ba-5ce3-4b8d-b3ea-7af8a3344591 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:20.365557 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a18ba-5ce3-4b8d-b3ea-7af8a3344591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:20.397038 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-567a18ba-5ce3-4b8d-b3ea-7af8a3344591 None None] MFA Rules not processed for user `7261ab46436e4546b8331aa8387cfa1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:20.398516 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-567a18ba-5ce3-4b8d-b3ea-7af8a3344591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:20.435496 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9cec6127-07c0-4569-81ab-bc56619f85ab admin admin] The token cache is being invalidate because project 3af4529ca03d410bb181575d9e4cd1d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:34:20.435837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9cec6127-07c0-4569-81ab-bc56619f85ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:20.436642 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9cec6127-07c0-4569-81ab-bc56619f85ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:20.440419 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 859/1723] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:34:20 2022] DELETE /identity/v3/projects/3af4529ca03d410bb181575d9e4cd1d6 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:34:20.441402 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec6127-07c0-4569-81ab-bc56619f85ab admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:20.447899 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9cec6127-07c0-4569-81ab-bc56619f85ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:20.488205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cec6127-07c0-4569-81ab-bc56619f85ab admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:20.498743 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3429a28-f6f9-4672-b5f0-08ab64345fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:20.499692 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3429a28-f6f9-4672-b5f0-08ab64345fd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:20.499976 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3429a28-f6f9-4672-b5f0-08ab64345fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:20.500245 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3429a28-f6f9-4672-b5f0-08ab64345fd7 admin admin] PATH_INFO: `/v3/projects/ba99c2cce3c14654861c7c3c1aa67f7f/users/941a1d616de0484d85aae59720e8b1c3/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:20.511045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 865/1724] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:20.514868 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a18ba-5ce3-4b8d-b3ea-7af8a3344591 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:20.515397 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-567a18ba-5ce3-4b8d-b3ea-7af8a3344591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:20.549952 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3429a28-f6f9-4672-b5f0-08ab64345fd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba99c2cce3c14654861c7c3c1aa67f7f, user_id=941a1d616de0484d85aae59720e8b1c3, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:20.552027 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3429a28-f6f9-4672-b5f0-08ab64345fd7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:20.565934 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a18ba-5ce3-4b8d-b3ea-7af8a3344591 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:20.584094 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca66811-9f77-4b10-a363-b807148227dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:20.584094 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca66811-9f77-4b10-a363-b807148227dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:20.584094 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca66811-9f77-4b10-a363-b807148227dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:20.584094 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca66811-9f77-4b10-a363-b807148227dc admin admin] PATH_INFO: `/v3/users/4575435437bb48aba67e0aeca7e5b707` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:20.584525 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca66811-9f77-4b10-a363-b807148227dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4575435437bb48aba67e0aeca7e5b707)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:20.586476 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca66811-9f77-4b10-a363-b807148227dc admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:20.651737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 860/1725] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:20 2022] PUT /identity/v3/projects/ba99c2cce3c14654861c7c3c1aa67f7f/users/941a1d616de0484d85aae59720e8b1c3/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:20.658580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3429a28-f6f9-4672-b5f0-08ab64345fd7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:20.799855 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3429a28-f6f9-4672-b5f0-08ab64345fd7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:20.809384 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c645db-2a5d-4e86-ba57-e62681986f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:20.811073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c645db-2a5d-4e86-ba57-e62681986f11 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:20.812485 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c645db-2a5d-4e86-ba57-e62681986f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:20.812999 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c645db-2a5d-4e86-ba57-e62681986f11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:20.821312 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c645db-2a5d-4e86-ba57-e62681986f11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:20.821312 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c645db-2a5d-4e86-ba57-e62681986f11 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:20.842527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 861/1726] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:20.846883 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c645db-2a5d-4e86-ba57-e62681986f11 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:20.849608 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4ca66811-9f77-4b10-a363-b807148227dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4575435437bb48aba67e0aeca7e5b707'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:20.857858 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4ca66811-9f77-4b10-a363-b807148227dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4575435437bb48aba67e0aeca7e5b707'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:20.870415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c645db-2a5d-4e86-ba57-e62681986f11 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:20.872140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4ca66811-9f77-4b10-a363-b807148227dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4575435437bb48aba67e0aeca7e5b707'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:20.879917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c14f8-b1d1-4274-b5a0-deb65e1b252e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:20.881027 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c14f8-b1d1-4274-b5a0-deb65e1b252e admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:20.881419 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c14f8-b1d1-4274-b5a0-deb65e1b252e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:20.881813 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c14f8-b1d1-4274-b5a0-deb65e1b252e admin admin] PATH_INFO: `/v3/projects/ba99c2cce3c14654861c7c3c1aa67f7f/users/941a1d616de0484d85aae59720e8b1c3/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:20.885283 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c14f8-b1d1-4274-b5a0-deb65e1b252e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba99c2cce3c14654861c7c3c1aa67f7f, user_id=941a1d616de0484d85aae59720e8b1c3, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:20.888418 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c14f8-b1d1-4274-b5a0-deb65e1b252e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:20.905237 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 862/1727] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:20 2022] PUT /identity/v3/projects/ba99c2cce3c14654861c7c3c1aa67f7f/users/941a1d616de0484d85aae59720e8b1c3/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:20.916928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9bc98-55a5-4cef-b3dc-e5836000f665 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:20.917516 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9bc98-55a5-4cef-b3dc-e5836000f665 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:20.917929 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9bc98-55a5-4cef-b3dc-e5836000f665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:20.954258 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-13e9bc98-55a5-4cef-b3dc-e5836000f665 None None] MFA Rules not processed for user `941a1d616de0484d85aae59720e8b1c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:20.955840 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-13e9bc98-55a5-4cef-b3dc-e5836000f665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:20.961071 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4ca66811-9f77-4b10-a363-b807148227dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4575435437bb48aba67e0aeca7e5b707'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:20.964209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4ca66811-9f77-4b10-a363-b807148227dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4575435437bb48aba67e0aeca7e5b707'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:21.016318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 866/1728] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:34:20 2022] DELETE /identity/v3/users/4575435437bb48aba67e0aeca7e5b707 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:21.093655 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 863/1729] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:20 2022] POST /identity/v3/auth/tokens => generated 3293 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:34:21.124899 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca66811-9f77-4b10-a363-b807148227dc admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:21.133092 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4ca66811-9f77-4b10-a363-b807148227dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:21.277587 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:21.283304 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca66811-9f77-4b10-a363-b807148227dc admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:21.288841 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f65cd-2fe3-4260-8696-cbf4f2d83d71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:21.290235 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f65cd-2fe3-4260-8696-cbf4f2d83d71 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:21.290622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f65cd-2fe3-4260-8696-cbf4f2d83d71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:21.291655 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f65cd-2fe3-4260-8696-cbf4f2d83d71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:21.402906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f65cd-2fe3-4260-8696-cbf4f2d83d71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:21.404573 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f65cd-2fe3-4260-8696-cbf4f2d83d71 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:21.418842 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 867/1730] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:21 2022] GET /identity/v3/auth/tokens => generated 3293 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:34:21.596611 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:21.766773 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e9bc98-55a5-4cef-b3dc-e5836000f665 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:21.831864 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f65cd-2fe3-4260-8696-cbf4f2d83d71 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:21.836527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-975f65cd-2fe3-4260-8696-cbf4f2d83d71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:21.858605 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e9bc98-55a5-4cef-b3dc-e5836000f665 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:21.864695 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791361ff-7135-486a-a8a8-32b01cc3012a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:21.867093 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791361ff-7135-486a-a8a8-32b01cc3012a service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:21.867093 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791361ff-7135-486a-a8a8-32b01cc3012a service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:21.867093 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791361ff-7135-486a-a8a8-32b01cc3012a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:21.868505 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-791361ff-7135-486a-a8a8-32b01cc3012a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:21.992679 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f65cd-2fe3-4260-8696-cbf4f2d83d71 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:22.004238 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e1d3d5-09ec-4248-bf5e-73e5f63e29c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:22.005644 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791361ff-7135-486a-a8a8-32b01cc3012a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:22.006113 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1d3d5-09ec-4248-bf5e-73e5f63e29c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:22.006713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1d3d5-09ec-4248-bf5e-73e5f63e29c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:22.007351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e1d3d5-09ec-4248-bf5e-73e5f63e29c7 admin admin] PATH_INFO: `/v3/projects/b9a0cbda2dc945ea9c009036bb59c48a` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:22.009107 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1d3d5-09ec-4248-bf5e-73e5f63e29c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9a0cbda2dc945ea9c009036bb59c48a)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:22.009473 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791361ff-7135-486a-a8a8-32b01cc3012a service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:22.012264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e1d3d5-09ec-4248-bf5e-73e5f63e29c7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:22.065258 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 864/1731] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:21 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:34:22.248744 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-10e1d3d5-09ec-4248-bf5e-73e5f63e29c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9a0cbda2dc945ea9c009036bb59c48a', 'request_id': 'req-10e1d3d5-09ec-4248-bf5e-73e5f63e29c7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:22.346025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-10e1d3d5-09ec-4248-bf5e-73e5f63e29c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9a0cbda2dc945ea9c009036bb59c48a', 'request_id': 'req-10e1d3d5-09ec-4248-bf5e-73e5f63e29c7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:22.354708 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-10e1d3d5-09ec-4248-bf5e-73e5f63e29c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9a0cbda2dc945ea9c009036bb59c48a', 'request_id': 'req-10e1d3d5-09ec-4248-bf5e-73e5f63e29c7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:22.368088 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-10e1d3d5-09ec-4248-bf5e-73e5f63e29c7 admin admin] The token cache is being invalidate because project b9a0cbda2dc945ea9c009036bb59c48a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:34:22.368452 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-10e1d3d5-09ec-4248-bf5e-73e5f63e29c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:22.369042 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-10e1d3d5-09ec-4248-bf5e-73e5f63e29c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:22.371319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 868/1732] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:34:21 2022] DELETE /identity/v3/projects/b9a0cbda2dc945ea9c009036bb59c48a => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:22.376266 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9558e02-340d-4e68-982e-51ae196fc0d4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:22.376804 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9558e02-340d-4e68-982e-51ae196fc0d4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:22.376804 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9558e02-340d-4e68-982e-51ae196fc0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:22.400656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d9558e02-340d-4e68-982e-51ae196fc0d4 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:22.402169 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d9558e02-340d-4e68-982e-51ae196fc0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:22.472507 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 865/1733] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:22.482162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61b255-9e81-4b81-8dca-87a4101df0f4 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:22.482477 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61b255-9e81-4b81-8dca-87a4101df0f4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:22.482788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a61b255-9e81-4b81-8dca-87a4101df0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:22.503024 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9a61b255-9e81-4b81-8dca-87a4101df0f4 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:22.504275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9a61b255-9e81-4b81-8dca-87a4101df0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:22.512965 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 869/1734] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:22.517815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9558e02-340d-4e68-982e-51ae196fc0d4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:22.528500 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9558e02-340d-4e68-982e-51ae196fc0d4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:22.533400 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131daf89-196b-4a8a-b032-2abefae0eef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:22.534089 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131daf89-196b-4a8a-b032-2abefae0eef2 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:22.534369 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131daf89-196b-4a8a-b032-2abefae0eef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:22.534642 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131daf89-196b-4a8a-b032-2abefae0eef2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:22.535400 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131daf89-196b-4a8a-b032-2abefae0eef2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:34:22.535740 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131daf89-196b-4a8a-b032-2abefae0eef2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:22.537441 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131daf89-196b-4a8a-b032-2abefae0eef2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:22.551496 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 866/1735] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:34:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:34:22.556815 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61b255-9e81-4b81-8dca-87a4101df0f4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:22.566920 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a61b255-9e81-4b81-8dca-87a4101df0f4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:22.571711 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62746ad6-c865-41e8-971d-b3aa7be43dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:22.572435 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62746ad6-c865-41e8-971d-b3aa7be43dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:22.572730 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62746ad6-c865-41e8-971d-b3aa7be43dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:22.573011 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62746ad6-c865-41e8-971d-b3aa7be43dc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:22.573718 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62746ad6-c865-41e8-971d-b3aa7be43dc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:22.582047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62746ad6-c865-41e8-971d-b3aa7be43dc0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:22.666150 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 870/1736] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:34:22 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:34:22.671753 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131daf89-196b-4a8a-b032-2abefae0eef2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:22.684111 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131daf89-196b-4a8a-b032-2abefae0eef2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:22.745431 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa1ef0-fed0-48ab-834f-1bc9ddd4b7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:22.746167 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa1ef0-fed0-48ab-834f-1bc9ddd4b7a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:22.746456 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa1ef0-fed0-48ab-834f-1bc9ddd4b7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:22.746730 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa1ef0-fed0-48ab-834f-1bc9ddd4b7a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:22.747435 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaa1ef0-fed0-48ab-834f-1bc9ddd4b7a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:22.749433 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaa1ef0-fed0-48ab-834f-1bc9ddd4b7a2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:22.855313 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 867/1737] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:34:22 2022] POST /identity/v3/users => generated 283 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:22.859992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62746ad6-c865-41e8-971d-b3aa7be43dc0 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:22.871323 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62746ad6-c865-41e8-971d-b3aa7be43dc0 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:22.875858 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755653d9-ade0-4219-9215-5b4ce141bda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:22.876517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755653d9-ade0-4219-9215-5b4ce141bda5 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:22.876795 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755653d9-ade0-4219-9215-5b4ce141bda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:22.877070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755653d9-ade0-4219-9215-5b4ce141bda5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:22.877818 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755653d9-ade0-4219-9215-5b4ce141bda5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:22.879611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755653d9-ade0-4219-9215-5b4ce141bda5 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:22.893925 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 871/1738] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:22.899054 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa1ef0-fed0-48ab-834f-1bc9ddd4b7a2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:22.911848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa1ef0-fed0-48ab-834f-1bc9ddd4b7a2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:22.917944 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9edf73-b645-45a5-afa7-16926eee4391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:22.918629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9edf73-b645-45a5-afa7-16926eee4391 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:22.918908 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9edf73-b645-45a5-afa7-16926eee4391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:22.919268 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9edf73-b645-45a5-afa7-16926eee4391 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:22.920112 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9edf73-b645-45a5-afa7-16926eee4391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:22.921712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9edf73-b645-45a5-afa7-16926eee4391 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:22.932878 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 868/1739] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:22.938722 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755653d9-ade0-4219-9215-5b4ce141bda5 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:22.948968 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755653d9-ade0-4219-9215-5b4ce141bda5 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:22.953525 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c122907-7b76-4885-873a-1b5fb6d88dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:22.954629 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c122907-7b76-4885-873a-1b5fb6d88dfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:22.954917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c122907-7b76-4885-873a-1b5fb6d88dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:22.955194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c122907-7b76-4885-873a-1b5fb6d88dfd admin admin] PATH_INFO: `/v3/projects/1981070ca7504a7a86cf866e06af4c9a/users/6d87c7426aa14694a7210207217bb733/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:22.972622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c122907-7b76-4885-873a-1b5fb6d88dfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1981070ca7504a7a86cf866e06af4c9a, user_id=6d87c7426aa14694a7210207217bb733, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:22.975447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c122907-7b76-4885-873a-1b5fb6d88dfd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:23.144697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 872/1740] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:22 2022] PUT /identity/v3/projects/1981070ca7504a7a86cf866e06af4c9a/users/6d87c7426aa14694a7210207217bb733/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:23.157525 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d187d-1c2a-4da1-bf2c-2d6c5516244c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:23.157863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d187d-1c2a-4da1-bf2c-2d6c5516244c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:23.158151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d187d-1c2a-4da1-bf2c-2d6c5516244c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:23.195381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-835d187d-1c2a-4da1-bf2c-2d6c5516244c None None] MFA Rules not processed for user `6d87c7426aa14694a7210207217bb733`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:23.198141 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-835d187d-1c2a-4da1-bf2c-2d6c5516244c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:23.331981 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 869/1741] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:23 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:34:23.339943 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c122907-7b76-4885-873a-1b5fb6d88dfd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:23.439141 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c122907-7b76-4885-873a-1b5fb6d88dfd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:23.444842 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea119c-9511-4f63-9ef3-13060d455b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:23.445721 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea119c-9511-4f63-9ef3-13060d455b92 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:23.446172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea119c-9511-4f63-9ef3-13060d455b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:23.446458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea119c-9511-4f63-9ef3-13060d455b92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:23.447356 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ea119c-9511-4f63-9ef3-13060d455b92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:23.450000 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ea119c-9511-4f63-9ef3-13060d455b92 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:23.548497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 873/1742] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:34:23 2022] POST /identity/v3/projects => generated 365 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:34:23.552847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d187d-1c2a-4da1-bf2c-2d6c5516244c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:23.567194 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d187d-1c2a-4da1-bf2c-2d6c5516244c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:23.632029 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372676a-404d-470e-8056-d2ce505a5ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:23.632770 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372676a-404d-470e-8056-d2ce505a5ebd admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:23.633056 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372676a-404d-470e-8056-d2ce505a5ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:23.633336 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372676a-404d-470e-8056-d2ce505a5ebd admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:23.634047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372676a-404d-470e-8056-d2ce505a5ebd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:23.636739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372676a-404d-470e-8056-d2ce505a5ebd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:23.639897 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:23.758468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 870/1743] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:34:23 2022] POST /identity/v3/users => generated 289 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:34:23.763696 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea119c-9511-4f63-9ef3-13060d455b92 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:23.775389 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea119c-9511-4f63-9ef3-13060d455b92 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:23.781204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff8f92-7539-4d32-a38e-65048c4aef16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:23.781884 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff8f92-7539-4d32-a38e-65048c4aef16 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:23.782325 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff8f92-7539-4d32-a38e-65048c4aef16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:23.782610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff8f92-7539-4d32-a38e-65048c4aef16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:23.783339 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff8f92-7539-4d32-a38e-65048c4aef16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:23.785140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff8f92-7539-4d32-a38e-65048c4aef16 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:23.797859 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 874/1744] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:23.803369 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372676a-404d-470e-8056-d2ce505a5ebd admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:23.814096 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372676a-404d-470e-8056-d2ce505a5ebd admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:23.819539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef976068-c21e-450e-b0d2-a97697cabe32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:23.820520 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef976068-c21e-450e-b0d2-a97697cabe32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:23.820849 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef976068-c21e-450e-b0d2-a97697cabe32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:23.821187 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef976068-c21e-450e-b0d2-a97697cabe32 admin admin] PATH_INFO: `/v3/projects/cf9a16b12c734de8b2233e3f2658a80d/users/01f925dc54754a94962f891bb1ef54b8/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:23.838926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef976068-c21e-450e-b0d2-a97697cabe32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf9a16b12c734de8b2233e3f2658a80d, user_id=01f925dc54754a94962f891bb1ef54b8, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:23.842610 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef976068-c21e-450e-b0d2-a97697cabe32 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:23.996153 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 871/1745] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:23 2022] PUT /identity/v3/projects/cf9a16b12c734de8b2233e3f2658a80d/users/01f925dc54754a94962f891bb1ef54b8/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:34:24.000816 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff8f92-7539-4d32-a38e-65048c4aef16 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:24.020216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff8f92-7539-4d32-a38e-65048c4aef16 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:24.085061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b25603-f5ad-4723-8bb1-3bb44c8e6c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:24.085656 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b25603-f5ad-4723-8bb1-3bb44c8e6c32 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:24.085969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b25603-f5ad-4723-8bb1-3bb44c8e6c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:24.086245 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b25603-f5ad-4723-8bb1-3bb44c8e6c32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:24.086982 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b25603-f5ad-4723-8bb1-3bb44c8e6c32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:24.089329 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b25603-f5ad-4723-8bb1-3bb44c8e6c32 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:24.101577 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 875/1746] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:24.108484 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef976068-c21e-450e-b0d2-a97697cabe32 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:24.121680 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef976068-c21e-450e-b0d2-a97697cabe32 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:24.127715 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c63299-19fa-42e5-9917-6c941a14bfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:24.128787 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c63299-19fa-42e5-9917-6c941a14bfaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:24.129205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c63299-19fa-42e5-9917-6c941a14bfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:24.129494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c63299-19fa-42e5-9917-6c941a14bfaf admin admin] PATH_INFO: `/v3/projects/cf9a16b12c734de8b2233e3f2658a80d/users/01f925dc54754a94962f891bb1ef54b8/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:24.131632 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c63299-19fa-42e5-9917-6c941a14bfaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf9a16b12c734de8b2233e3f2658a80d, user_id=01f925dc54754a94962f891bb1ef54b8, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:24.134368 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c63299-19fa-42e5-9917-6c941a14bfaf admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:24.146641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 872/1747] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:24 2022] PUT /identity/v3/projects/cf9a16b12c734de8b2233e3f2658a80d/users/01f925dc54754a94962f891bb1ef54b8/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:24.157420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17358c4-113e-41b1-ae93-556cd255842c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:24.157720 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17358c4-113e-41b1-ae93-556cd255842c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:24.158042 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17358c4-113e-41b1-ae93-556cd255842c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:24.207318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c17358c4-113e-41b1-ae93-556cd255842c None None] MFA Rules not processed for user `01f925dc54754a94962f891bb1ef54b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:24.209661 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c17358c4-113e-41b1-ae93-556cd255842c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:24.292863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 876/1748] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:24 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:34:24.305103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c63299-19fa-42e5-9917-6c941a14bfaf admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:24.308447 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-11c63299-19fa-42e5-9917-6c941a14bfaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:24.379684 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c63299-19fa-42e5-9917-6c941a14bfaf admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:24.386393 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c70f531-085b-43c7-87a4-d392553065b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:24.386393 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70f531-085b-43c7-87a4-d392553065b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:24.386393 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70f531-085b-43c7-87a4-d392553065b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:24.386393 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70f531-085b-43c7-87a4-d392553065b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:24.395985 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c70f531-085b-43c7-87a4-d392553065b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:24.397549 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c70f531-085b-43c7-87a4-d392553065b4 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:24.409539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 873/1749] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:24 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:34:24.413476 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:24.730678 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17358c4-113e-41b1-ae93-556cd255842c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:24.734376 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c17358c4-113e-41b1-ae93-556cd255842c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:24.748156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17358c4-113e-41b1-ae93-556cd255842c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:24.751932 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb4fc91-6df6-40e1-8cb5-f8da932101a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:24.752837 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4fc91-6df6-40e1-8cb5-f8da932101a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:24.753272 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4fc91-6df6-40e1-8cb5-f8da932101a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:24.753573 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4fc91-6df6-40e1-8cb5-f8da932101a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:24.816300 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb4fc91-6df6-40e1-8cb5-f8da932101a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:24.817804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb4fc91-6df6-40e1-8cb5-f8da932101a9 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:24.827722 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 877/1750] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:24 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:34:24.855624 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:26.212567 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:27.131867 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c70f531-085b-43c7-87a4-d392553065b4 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:27.133558 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8c70f531-085b-43c7-87a4-d392553065b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:27.240312 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c70f531-085b-43c7-87a4-d392553065b4 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:27.249031 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29691be4-8b4f-4b70-8b8c-13d611dea517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:27.250692 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29691be4-8b4f-4b70-8b8c-13d611dea517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:27.251279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29691be4-8b4f-4b70-8b8c-13d611dea517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:27.251956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29691be4-8b4f-4b70-8b8c-13d611dea517 admin admin] PATH_INFO: `/v3/users/d8c444b440324b858ed8cd3f30924c36` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:27.260954 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29691be4-8b4f-4b70-8b8c-13d611dea517 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8c444b440324b858ed8cd3f30924c36)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:27.260954 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29691be4-8b4f-4b70-8b8c-13d611dea517 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:27.527438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:27.539103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-29691be4-8b4f-4b70-8b8c-13d611dea517 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8c444b440324b858ed8cd3f30924c36'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:27.539857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-29691be4-8b4f-4b70-8b8c-13d611dea517 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8c444b440324b858ed8cd3f30924c36'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:27.635808 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:27.637128 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-29691be4-8b4f-4b70-8b8c-13d611dea517 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8c444b440324b858ed8cd3f30924c36'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:27.648374 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-29691be4-8b4f-4b70-8b8c-13d611dea517 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8c444b440324b858ed8cd3f30924c36'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:27.649500 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-29691be4-8b4f-4b70-8b8c-13d611dea517 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8c444b440324b858ed8cd3f30924c36'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:27.679566 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 874/1751] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:34:27 2022] DELETE /identity/v3/users/d8c444b440324b858ed8cd3f30924c36 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:27.686216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ebedf-1051-4bb1-9dbb-cd2546fb61be None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:27.687197 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ebedf-1051-4bb1-9dbb-cd2546fb61be None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:27.688743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227ebedf-1051-4bb1-9dbb-cd2546fb61be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:27.723245 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-227ebedf-1051-4bb1-9dbb-cd2546fb61be None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:27.725647 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-227ebedf-1051-4bb1-9dbb-cd2546fb61be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:27.903665 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 878/1752] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:34:27.910993 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29691be4-8b4f-4b70-8b8c-13d611dea517 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:27.914366 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-29691be4-8b4f-4b70-8b8c-13d611dea517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:28.131608 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29691be4-8b4f-4b70-8b8c-13d611dea517 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:28.141344 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a1a88-7df3-454f-a508-2e7965ec614a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:28.147933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a1a88-7df3-454f-a508-2e7965ec614a service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:28.152221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a1a88-7df3-454f-a508-2e7965ec614a service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:28.152662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a1a88-7df3-454f-a508-2e7965ec614a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:28.169545 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a1a88-7df3-454f-a508-2e7965ec614a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:28.172062 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a1a88-7df3-454f-a508-2e7965ec614a service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:28.187965 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 875/1753] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:34:29.598332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227ebedf-1051-4bb1-9dbb-cd2546fb61be None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:29.600280 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-227ebedf-1051-4bb1-9dbb-cd2546fb61be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:29.622029 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227ebedf-1051-4bb1-9dbb-cd2546fb61be None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:29.656778 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f140237-bd32-4e58-954b-ccfb31c0f645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:29.658250 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f140237-bd32-4e58-954b-ccfb31c0f645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:29.659820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f140237-bd32-4e58-954b-ccfb31c0f645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:29.660538 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f140237-bd32-4e58-954b-ccfb31c0f645 admin admin] PATH_INFO: `/v3/projects/bbc14b32ce7b47d18c84f545cb65b732` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:29.666466 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f140237-bd32-4e58-954b-ccfb31c0f645 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbc14b32ce7b47d18c84f545cb65b732)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:29.679003 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f140237-bd32-4e58-954b-ccfb31c0f645 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:30.070608 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5f140237-bd32-4e58-954b-ccfb31c0f645 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbc14b32ce7b47d18c84f545cb65b732', 'request_id': 'req-5f140237-bd32-4e58-954b-ccfb31c0f645'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:30.236414 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5f140237-bd32-4e58-954b-ccfb31c0f645 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbc14b32ce7b47d18c84f545cb65b732', 'request_id': 'req-5f140237-bd32-4e58-954b-ccfb31c0f645'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:30.251795 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5f140237-bd32-4e58-954b-ccfb31c0f645 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbc14b32ce7b47d18c84f545cb65b732', 'request_id': 'req-5f140237-bd32-4e58-954b-ccfb31c0f645'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:30.297855 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5f140237-bd32-4e58-954b-ccfb31c0f645 admin admin] The token cache is being invalidate because project bbc14b32ce7b47d18c84f545cb65b732 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:34:30.301768 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5f140237-bd32-4e58-954b-ccfb31c0f645 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:30.306035 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5f140237-bd32-4e58-954b-ccfb31c0f645 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:30.315505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 879/1754] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:34:29 2022] DELETE /identity/v3/projects/bbc14b32ce7b47d18c84f545cb65b732 => generated 0 bytes in 718 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:30.322505 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a1a88-7df3-454f-a508-2e7965ec614a service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:30.380686 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a1a88-7df3-454f-a508-2e7965ec614a service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:30.524426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2d51b-33a4-4041-b02f-fadd1c38b588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:30.525728 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2d51b-33a4-4041-b02f-fadd1c38b588 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:30.526204 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2d51b-33a4-4041-b02f-fadd1c38b588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:30.528389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac2d51b-33a4-4041-b02f-fadd1c38b588 admin admin] PATH_INFO: `/v3/users/941a1d616de0484d85aae59720e8b1c3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:30.530135 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2d51b-33a4-4041-b02f-fadd1c38b588 admin admin] RBAC: Authorizing `identity:delete_user(user_id=941a1d616de0484d85aae59720e8b1c3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:30.532451 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac2d51b-33a4-4041-b02f-fadd1c38b588 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:30.854462 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fac2d51b-33a4-4041-b02f-fadd1c38b588 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '941a1d616de0484d85aae59720e8b1c3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:30.855421 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fac2d51b-33a4-4041-b02f-fadd1c38b588 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '941a1d616de0484d85aae59720e8b1c3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:31.005853 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fac2d51b-33a4-4041-b02f-fadd1c38b588 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '941a1d616de0484d85aae59720e8b1c3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:31.012588 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:31.033459 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fac2d51b-33a4-4041-b02f-fadd1c38b588 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '941a1d616de0484d85aae59720e8b1c3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:31.035113 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fac2d51b-33a4-4041-b02f-fadd1c38b588 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '941a1d616de0484d85aae59720e8b1c3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:31.055273 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91783a-a793-4df8-8f7c-aa70870e785d None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:31.056143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91783a-a793-4df8-8f7c-aa70870e785d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:31.056523 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91783a-a793-4df8-8f7c-aa70870e785d None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:31.062120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 880/1755] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 30 16:34:31 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:34:31.077811 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ff89d-e5fa-48d8-be96-8f8806219c54 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:31.078689 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ff89d-e5fa-48d8-be96-8f8806219c54 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:31.079398 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ff89d-e5fa-48d8-be96-8f8806219c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:31.108740 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 876/1756] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:34:30 2022] DELETE /identity/v3/users/941a1d616de0484d85aae59720e8b1c3 => generated 0 bytes in 786 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:34:31.138685 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-be0ff89d-e5fa-48d8-be96-8f8806219c54 None None] MFA Rules not processed for user `71cca3708386459197abf6c67505bc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:31.144697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-be0ff89d-e5fa-48d8-be96-8f8806219c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:31.360704 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:31.385925 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 881/1757] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 30 16:34:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:31.480546 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2d51b-33a4-4041-b02f-fadd1c38b588 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:31.482535 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fac2d51b-33a4-4041-b02f-fadd1c38b588 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:31.509105 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac2d51b-33a4-4041-b02f-fadd1c38b588 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:31.530055 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ac4a5-f29a-45a1-b805-15398e3db17e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:31.532579 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ac4a5-f29a-45a1-b805-15398e3db17e service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:31.533495 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ac4a5-f29a-45a1-b805-15398e3db17e service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:31.534489 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ac4a5-f29a-45a1-b805-15398e3db17e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:31.560379 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ac4a5-f29a-45a1-b805-15398e3db17e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:31.566556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ac4a5-f29a-45a1-b805-15398e3db17e service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:31.604599 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 877/1758] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:34:32.874809 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ff89d-e5fa-48d8-be96-8f8806219c54 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:32.878267 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-be0ff89d-e5fa-48d8-be96-8f8806219c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:33.203178 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ff89d-e5fa-48d8-be96-8f8806219c54 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:33.236774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b802e4-f62f-4927-ba43-f1bda6e8a343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:33.247852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b802e4-f62f-4927-ba43-f1bda6e8a343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:33.247852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b802e4-f62f-4927-ba43-f1bda6e8a343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:33.247852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b802e4-f62f-4927-ba43-f1bda6e8a343 admin admin] PATH_INFO: `/v3/projects/ba99c2cce3c14654861c7c3c1aa67f7f` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:33.247852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b802e4-f62f-4927-ba43-f1bda6e8a343 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba99c2cce3c14654861c7c3c1aa67f7f)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:33.254657 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b802e4-f62f-4927-ba43-f1bda6e8a343 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:33.633833 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-12b802e4-f62f-4927-ba43-f1bda6e8a343 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba99c2cce3c14654861c7c3c1aa67f7f', 'request_id': 'req-12b802e4-f62f-4927-ba43-f1bda6e8a343'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:33.751315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-12b802e4-f62f-4927-ba43-f1bda6e8a343 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba99c2cce3c14654861c7c3c1aa67f7f', 'request_id': 'req-12b802e4-f62f-4927-ba43-f1bda6e8a343'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:33.762298 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-12b802e4-f62f-4927-ba43-f1bda6e8a343 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba99c2cce3c14654861c7c3c1aa67f7f', 'request_id': 'req-12b802e4-f62f-4927-ba43-f1bda6e8a343'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:33.790435 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-12b802e4-f62f-4927-ba43-f1bda6e8a343 admin admin] The token cache is being invalidate because project ba99c2cce3c14654861c7c3c1aa67f7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:34:33.790996 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-12b802e4-f62f-4927-ba43-f1bda6e8a343 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:33.795738 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-12b802e4-f62f-4927-ba43-f1bda6e8a343 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:33.798427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 882/1759] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:34:32 2022] DELETE /identity/v3/projects/ba99c2cce3c14654861c7c3c1aa67f7f => generated 0 bytes in 924 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:33.804972 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b1ee5-d5ca-4d43-abbb-a278bc7d14d7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:33.805893 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b1ee5-d5ca-4d43-abbb-a278bc7d14d7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:33.806329 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b1ee5-d5ca-4d43-abbb-a278bc7d14d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:33.848826 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a91b1ee5-d5ca-4d43-abbb-a278bc7d14d7 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:33.850670 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a91b1ee5-d5ca-4d43-abbb-a278bc7d14d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:34.002232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 878/1760] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:34.006705 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0110d4-e9bd-4a47-9ebe-02af6a8f46b8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:34.006705 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0110d4-e9bd-4a47-9ebe-02af6a8f46b8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:34.007138 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0110d4-e9bd-4a47-9ebe-02af6a8f46b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:34.048892 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ba0110d4-e9bd-4a47-9ebe-02af6a8f46b8 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:34.051148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ba0110d4-e9bd-4a47-9ebe-02af6a8f46b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:34.071552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 883/1761] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:34.074919 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b1ee5-d5ca-4d43-abbb-a278bc7d14d7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:34.095284 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b1ee5-d5ca-4d43-abbb-a278bc7d14d7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:34.102736 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2434ca-0133-44a2-83de-05bc42b5a452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:34.119220 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2434ca-0133-44a2-83de-05bc42b5a452 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:34.119797 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2434ca-0133-44a2-83de-05bc42b5a452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:34.120237 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2434ca-0133-44a2-83de-05bc42b5a452 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:34.121765 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2434ca-0133-44a2-83de-05bc42b5a452 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:34:34.124282 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2434ca-0133-44a2-83de-05bc42b5a452 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:34.138236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2434ca-0133-44a2-83de-05bc42b5a452 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:34.172605 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 879/1762] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:34:34.178942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0110d4-e9bd-4a47-9ebe-02af6a8f46b8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:34.209617 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0110d4-e9bd-4a47-9ebe-02af6a8f46b8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:34.225130 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f05b3e-005e-4e88-87b3-b4674dd729c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:34.225634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f05b3e-005e-4e88-87b3-b4674dd729c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:34.226662 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f05b3e-005e-4e88-87b3-b4674dd729c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:34.227553 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f05b3e-005e-4e88-87b3-b4674dd729c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:34.234794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f05b3e-005e-4e88-87b3-b4674dd729c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:34.240284 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f05b3e-005e-4e88-87b3-b4674dd729c7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:34.409444 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 884/1763] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:34:34 2022] POST /identity/v3/projects => generated 387 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:34.414964 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2434ca-0133-44a2-83de-05bc42b5a452 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:34.433038 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2434ca-0133-44a2-83de-05bc42b5a452 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:34.609204 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92192c6c-01c9-48e3-b848-60256f9f3446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:34.610839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92192c6c-01c9-48e3-b848-60256f9f3446 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:34.611131 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92192c6c-01c9-48e3-b848-60256f9f3446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:34.618664 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92192c6c-01c9-48e3-b848-60256f9f3446 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:34.618664 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92192c6c-01c9-48e3-b848-60256f9f3446 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:34.618664 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92192c6c-01c9-48e3-b848-60256f9f3446 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:34.817394 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f05b3e-005e-4e88-87b3-b4674dd729c7 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:34.834433 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 880/1764] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:34:34 2022] POST /identity/v3/users => generated 294 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:34.839411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f05b3e-005e-4e88-87b3-b4674dd729c7 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:34.846962 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd52229-e5df-4548-bdc9-28d099ccaa0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:34.848233 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd52229-e5df-4548-bdc9-28d099ccaa0b admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:34.848696 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd52229-e5df-4548-bdc9-28d099ccaa0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:34.849124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd52229-e5df-4548-bdc9-28d099ccaa0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:34.850448 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd52229-e5df-4548-bdc9-28d099ccaa0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:34.852565 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd52229-e5df-4548-bdc9-28d099ccaa0b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:34.879174 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 885/1765] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:34:34.886468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92192c6c-01c9-48e3-b848-60256f9f3446 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:34.908754 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92192c6c-01c9-48e3-b848-60256f9f3446 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:34.914823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7c8b8d-2f6f-4f00-a9c5-19528d4914af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:34.915845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c8b8d-2f6f-4f00-a9c5-19528d4914af admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:34.918852 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c8b8d-2f6f-4f00-a9c5-19528d4914af admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:34.918852 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7c8b8d-2f6f-4f00-a9c5-19528d4914af admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:34.918852 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7c8b8d-2f6f-4f00-a9c5-19528d4914af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:34.928981 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7c8b8d-2f6f-4f00-a9c5-19528d4914af admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:34.964151 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd52229-e5df-4548-bdc9-28d099ccaa0b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:34.965096 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 881/1766] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:34.984766 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd52229-e5df-4548-bdc9-28d099ccaa0b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:35.011135 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b1b84-1883-4d5d-82ad-c18039d3aef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:35.012127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b1b84-1883-4d5d-82ad-c18039d3aef9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:35.013964 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b1b84-1883-4d5d-82ad-c18039d3aef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:35.014318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b1b84-1883-4d5d-82ad-c18039d3aef9 admin admin] PATH_INFO: `/v3/projects/64c43c586a7b4e7eab7c4a0bd59e5eba/users/d44b9525cbab46f9b9e8b12ea2352532/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:35.039365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b1b84-1883-4d5d-82ad-c18039d3aef9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64c43c586a7b4e7eab7c4a0bd59e5eba, user_id=d44b9525cbab46f9b9e8b12ea2352532, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:35.043218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b1b84-1883-4d5d-82ad-c18039d3aef9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:35.223806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 886/1767] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:34 2022] PUT /identity/v3/projects/64c43c586a7b4e7eab7c4a0bd59e5eba/users/d44b9525cbab46f9b9e8b12ea2352532/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:35.235260 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36711346-59dd-44a0-905d-20106d376928 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:35.239892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36711346-59dd-44a0-905d-20106d376928 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:35.239892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36711346-59dd-44a0-905d-20106d376928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:35.290997 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-36711346-59dd-44a0-905d-20106d376928 None None] MFA Rules not processed for user `d44b9525cbab46f9b9e8b12ea2352532`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:35.293901 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-36711346-59dd-44a0-905d-20106d376928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:35.315987 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:35.489205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 882/1768] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:35.493767 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b1b84-1883-4d5d-82ad-c18039d3aef9 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:35.547968 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:35.626337 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b1b84-1883-4d5d-82ad-c18039d3aef9 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:35.631408 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d6de5-9db9-4a4a-8df9-317c73cf49e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:35.632130 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d6de5-9db9-4a4a-8df9-317c73cf49e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:35.632415 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d6de5-9db9-4a4a-8df9-317c73cf49e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:35.632691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139d6de5-9db9-4a4a-8df9-317c73cf49e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:35.633418 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d6de5-9db9-4a4a-8df9-317c73cf49e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:35.635624 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139d6de5-9db9-4a4a-8df9-317c73cf49e9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:35.709882 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36711346-59dd-44a0-905d-20106d376928 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:35.711587 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-36711346-59dd-44a0-905d-20106d376928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:35.736613 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36711346-59dd-44a0-905d-20106d376928 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:35.743157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101015b5-7980-4c59-937e-80912f3eaaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:35.744381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101015b5-7980-4c59-937e-80912f3eaaf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:35.745047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101015b5-7980-4c59-937e-80912f3eaaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:35.745333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101015b5-7980-4c59-937e-80912f3eaaf1 admin admin] PATH_INFO: `/v3/users/6d87c7426aa14694a7210207217bb733` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:35.746617 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101015b5-7980-4c59-937e-80912f3eaaf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d87c7426aa14694a7210207217bb733)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:35.749016 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101015b5-7980-4c59-937e-80912f3eaaf1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:35.796757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 887/1769] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:34:35 2022] POST /identity/v3/projects => generated 387 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:34:35.801312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d6de5-9db9-4a4a-8df9-317c73cf49e9 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:35.967029 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139d6de5-9db9-4a4a-8df9-317c73cf49e9 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:35.974809 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ce2ff-3170-4528-ac7a-21e5560854c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:35.975905 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ce2ff-3170-4528-ac7a-21e5560854c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:35.976337 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ce2ff-3170-4528-ac7a-21e5560854c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:35.976805 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ce2ff-3170-4528-ac7a-21e5560854c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:35.977658 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ce2ff-3170-4528-ac7a-21e5560854c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:35.981152 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3ce2ff-3170-4528-ac7a-21e5560854c1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:36.066937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-101015b5-7980-4c59-937e-80912f3eaaf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d87c7426aa14694a7210207217bb733'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:36.068543 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-101015b5-7980-4c59-937e-80912f3eaaf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d87c7426aa14694a7210207217bb733'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:36.091885 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 888/1770] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:34:35 2022] POST /identity/v3/users => generated 300 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:36.096961 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ce2ff-3170-4528-ac7a-21e5560854c1 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:36.117846 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ce2ff-3170-4528-ac7a-21e5560854c1 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:36.134344 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484024fb-803f-43ff-957d-1f32dc147fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:36.135046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484024fb-803f-43ff-957d-1f32dc147fce admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:36.135661 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484024fb-803f-43ff-957d-1f32dc147fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:36.136101 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484024fb-803f-43ff-957d-1f32dc147fce admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:36.137104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484024fb-803f-43ff-957d-1f32dc147fce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:36.141121 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484024fb-803f-43ff-957d-1f32dc147fce admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:36.165972 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-101015b5-7980-4c59-937e-80912f3eaaf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d87c7426aa14694a7210207217bb733'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:36.167700 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 889/1771] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:36.170972 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484024fb-803f-43ff-957d-1f32dc147fce admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:36.180395 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-101015b5-7980-4c59-937e-80912f3eaaf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d87c7426aa14694a7210207217bb733'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:36.181558 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-101015b5-7980-4c59-937e-80912f3eaaf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d87c7426aa14694a7210207217bb733'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:36.187490 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484024fb-803f-43ff-957d-1f32dc147fce admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:36.194214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f582d2-5c5d-4896-92d7-44dcfdb21db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:36.195365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f582d2-5c5d-4896-92d7-44dcfdb21db6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:36.195825 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f582d2-5c5d-4896-92d7-44dcfdb21db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:36.196196 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f582d2-5c5d-4896-92d7-44dcfdb21db6 admin admin] PATH_INFO: `/v3/projects/1cf8f499cd9f49a49f82b06c0b14c436/users/110afb9f6c39468aa31830f072ed5c91/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:36.209539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 883/1772] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:34:35 2022] DELETE /identity/v3/users/6d87c7426aa14694a7210207217bb733 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:34:36.214223 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f1fc4-f06e-49ab-ac95-530fdd36db02 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:36.214588 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f1fc4-f06e-49ab-ac95-530fdd36db02 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:36.215375 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482f1fc4-f06e-49ab-ac95-530fdd36db02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:36.223955 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f582d2-5c5d-4896-92d7-44dcfdb21db6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cf8f499cd9f49a49f82b06c0b14c436, user_id=110afb9f6c39468aa31830f072ed5c91, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:36.228111 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f582d2-5c5d-4896-92d7-44dcfdb21db6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:36.254266 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-482f1fc4-f06e-49ab-ac95-530fdd36db02 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:36.255752 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-482f1fc4-f06e-49ab-ac95-530fdd36db02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:36.332882 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 884/1773] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:34:36.343928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f1fc4-f06e-49ab-ac95-530fdd36db02 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:36.345451 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-482f1fc4-f06e-49ab-ac95-530fdd36db02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:36.368040 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 890/1774] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:36 2022] PUT /identity/v3/projects/1cf8f499cd9f49a49f82b06c0b14c436/users/110afb9f6c39468aa31830f072ed5c91/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:36.377256 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f582d2-5c5d-4896-92d7-44dcfdb21db6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:36.379506 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a4f582d2-5c5d-4896-92d7-44dcfdb21db6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:36.410681 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482f1fc4-f06e-49ab-ac95-530fdd36db02 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:36.491610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f582d2-5c5d-4896-92d7-44dcfdb21db6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:36.501380 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e9e9ba-c845-4aa7-8f16-44b3a7ed5bdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:36.502522 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9e9ba-c845-4aa7-8f16-44b3a7ed5bdc service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:36.503039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9e9ba-c845-4aa7-8f16-44b3a7ed5bdc service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:36.503668 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e9e9ba-c845-4aa7-8f16-44b3a7ed5bdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:36.513697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2d377-39d3-4d12-ade1-d79a484b06ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:36.514935 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2d377-39d3-4d12-ade1-d79a484b06ed admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:36.514935 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2d377-39d3-4d12-ade1-d79a484b06ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:36.514935 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2d377-39d3-4d12-ade1-d79a484b06ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:36.516452 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2d377-39d3-4d12-ade1-d79a484b06ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:36.520469 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2d377-39d3-4d12-ade1-d79a484b06ed admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:36.526432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e9e9ba-c845-4aa7-8f16-44b3a7ed5bdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:36.528564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e9e9ba-c845-4aa7-8f16-44b3a7ed5bdc service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:36.538516 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 891/1775] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:36.540607 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2d377-39d3-4d12-ade1-d79a484b06ed admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:36.553752 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2d377-39d3-4d12-ade1-d79a484b06ed admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:36.566766 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 885/1776] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:34:36.568530 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e5a66-164d-4639-a0f0-fdfae737fa05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:36.569718 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e5a66-164d-4639-a0f0-fdfae737fa05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:36.570104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e5a66-164d-4639-a0f0-fdfae737fa05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:36.570476 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48e5a66-164d-4639-a0f0-fdfae737fa05 admin admin] PATH_INFO: `/v3/projects/1cf8f499cd9f49a49f82b06c0b14c436/users/110afb9f6c39468aa31830f072ed5c91/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:36.573159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e5a66-164d-4639-a0f0-fdfae737fa05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cf8f499cd9f49a49f82b06c0b14c436, user_id=110afb9f6c39468aa31830f072ed5c91, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:36.576659 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48e5a66-164d-4639-a0f0-fdfae737fa05 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:36.590905 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 892/1777] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:36 2022] PUT /identity/v3/projects/1cf8f499cd9f49a49f82b06c0b14c436/users/110afb9f6c39468aa31830f072ed5c91/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:36.603590 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a796f3-f3c6-4fd8-8fca-23f13cfdec89 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:36.604106 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a796f3-f3c6-4fd8-8fca-23f13cfdec89 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:36.604602 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a796f3-f3c6-4fd8-8fca-23f13cfdec89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:36.656306 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-61a796f3-f3c6-4fd8-8fca-23f13cfdec89 None None] MFA Rules not processed for user `110afb9f6c39468aa31830f072ed5c91`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:36.656947 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-61a796f3-f3c6-4fd8-8fca-23f13cfdec89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:36.775021 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 886/1778] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:36.796365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e5a66-164d-4639-a0f0-fdfae737fa05 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:36.799052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d48e5a66-164d-4639-a0f0-fdfae737fa05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:36.890969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48e5a66-164d-4639-a0f0-fdfae737fa05 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:36.896203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffecaac-fb07-4f67-8a53-6140f2196106 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:36.896886 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffecaac-fb07-4f67-8a53-6140f2196106 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:36.897166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffecaac-fb07-4f67-8a53-6140f2196106 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:36.897493 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffecaac-fb07-4f67-8a53-6140f2196106 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:36.908580 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffecaac-fb07-4f67-8a53-6140f2196106 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:36.909681 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffecaac-fb07-4f67-8a53-6140f2196106 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:36.923787 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 893/1779] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:34:36.960317 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a796f3-f3c6-4fd8-8fca-23f13cfdec89 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:37.041759 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a796f3-f3c6-4fd8-8fca-23f13cfdec89 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:37.046897 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe56bf3e-bf2f-4297-b404-3f7803e8a7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:37.048021 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56bf3e-bf2f-4297-b404-3f7803e8a7b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:37.048532 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56bf3e-bf2f-4297-b404-3f7803e8a7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:37.048532 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56bf3e-bf2f-4297-b404-3f7803e8a7b9 admin admin] PATH_INFO: `/v3/users/d44b9525cbab46f9b9e8b12ea2352532` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:37.049922 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe56bf3e-bf2f-4297-b404-3f7803e8a7b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d44b9525cbab46f9b9e8b12ea2352532)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:37.052610 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe56bf3e-bf2f-4297-b404-3f7803e8a7b9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:37.369443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe56bf3e-bf2f-4297-b404-3f7803e8a7b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd44b9525cbab46f9b9e8b12ea2352532'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:37.372371 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe56bf3e-bf2f-4297-b404-3f7803e8a7b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd44b9525cbab46f9b9e8b12ea2352532'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:37.416857 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffecaac-fb07-4f67-8a53-6140f2196106 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:37.421365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bffecaac-fb07-4f67-8a53-6140f2196106 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:37.466759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffecaac-fb07-4f67-8a53-6140f2196106 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:37.476272 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3918d-3915-40de-af49-36c882fa2145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:37.479748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3918d-3915-40de-af49-36c882fa2145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:37.480093 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3918d-3915-40de-af49-36c882fa2145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:37.480568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3918d-3915-40de-af49-36c882fa2145 admin admin] PATH_INFO: `/v3/projects/1981070ca7504a7a86cf866e06af4c9a` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:37.482743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e3918d-3915-40de-af49-36c882fa2145 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1981070ca7504a7a86cf866e06af4c9a)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:37.502975 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e3918d-3915-40de-af49-36c882fa2145 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:37.526608 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe56bf3e-bf2f-4297-b404-3f7803e8a7b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd44b9525cbab46f9b9e8b12ea2352532'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:37.541733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe56bf3e-bf2f-4297-b404-3f7803e8a7b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd44b9525cbab46f9b9e8b12ea2352532'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:37.542887 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe56bf3e-bf2f-4297-b404-3f7803e8a7b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd44b9525cbab46f9b9e8b12ea2352532'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:37.573531 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 887/1780] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:34:36 2022] DELETE /identity/v3/users/d44b9525cbab46f9b9e8b12ea2352532 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:37.578547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225a9ed-46b8-4baa-8330-546f05186e97 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:37.578717 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225a9ed-46b8-4baa-8330-546f05186e97 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:37.579541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225a9ed-46b8-4baa-8330-546f05186e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:37.605673 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b225a9ed-46b8-4baa-8330-546f05186e97 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:37.610442 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b225a9ed-46b8-4baa-8330-546f05186e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:37.703187 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 888/1781] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:37.711092 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225a9ed-46b8-4baa-8330-546f05186e97 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:37.712707 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b225a9ed-46b8-4baa-8330-546f05186e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:37.715072 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a4e3918d-3915-40de-af49-36c882fa2145 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1981070ca7504a7a86cf866e06af4c9a', 'request_id': 'req-a4e3918d-3915-40de-af49-36c882fa2145'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:37.822116 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225a9ed-46b8-4baa-8330-546f05186e97 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:37.882868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57278874-852c-4e28-bc46-630d4dfa0986 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:37.883614 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57278874-852c-4e28-bc46-630d4dfa0986 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:37.883978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57278874-852c-4e28-bc46-630d4dfa0986 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:37.889191 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57278874-852c-4e28-bc46-630d4dfa0986 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:37.889405 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a4e3918d-3915-40de-af49-36c882fa2145 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1981070ca7504a7a86cf866e06af4c9a', 'request_id': 'req-a4e3918d-3915-40de-af49-36c882fa2145'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:37.901071 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a4e3918d-3915-40de-af49-36c882fa2145 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1981070ca7504a7a86cf866e06af4c9a', 'request_id': 'req-a4e3918d-3915-40de-af49-36c882fa2145'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:37.923909 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a4e3918d-3915-40de-af49-36c882fa2145 admin admin] The token cache is being invalidate because project 1981070ca7504a7a86cf866e06af4c9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:34:37.924513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a4e3918d-3915-40de-af49-36c882fa2145 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:37.925564 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a4e3918d-3915-40de-af49-36c882fa2145 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:37.928083 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 894/1782] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:34:37 2022] DELETE /identity/v3/projects/1981070ca7504a7a86cf866e06af4c9a => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:34:37.934278 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3918d-3915-40de-af49-36c882fa2145 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:37.937702 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a4e3918d-3915-40de-af49-36c882fa2145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:37.956976 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57278874-852c-4e28-bc46-630d4dfa0986 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:37.958564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57278874-852c-4e28-bc46-630d4dfa0986 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:37.970788 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 889/1783] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:34:38.082129 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3918d-3915-40de-af49-36c882fa2145 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:38.090356 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45623483-ccc0-4ca5-82ae-72a619023460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:38.091636 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45623483-ccc0-4ca5-82ae-72a619023460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:38.092061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45623483-ccc0-4ca5-82ae-72a619023460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:38.092602 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45623483-ccc0-4ca5-82ae-72a619023460 admin admin] PATH_INFO: `/v3/users/01f925dc54754a94962f891bb1ef54b8` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:38.095176 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45623483-ccc0-4ca5-82ae-72a619023460 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01f925dc54754a94962f891bb1ef54b8)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:38.099067 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45623483-ccc0-4ca5-82ae-72a619023460 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:38.521756 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-45623483-ccc0-4ca5-82ae-72a619023460 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01f925dc54754a94962f891bb1ef54b8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:38.524154 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-45623483-ccc0-4ca5-82ae-72a619023460 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01f925dc54754a94962f891bb1ef54b8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:38.571780 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-45623483-ccc0-4ca5-82ae-72a619023460 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01f925dc54754a94962f891bb1ef54b8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:38.696705 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-45623483-ccc0-4ca5-82ae-72a619023460 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01f925dc54754a94962f891bb1ef54b8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:38.704828 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-45623483-ccc0-4ca5-82ae-72a619023460 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01f925dc54754a94962f891bb1ef54b8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:38.745640 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 895/1784] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:34:37 2022] DELETE /identity/v3/users/01f925dc54754a94962f891bb1ef54b8 => generated 0 bytes in 805 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:39.205134 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57278874-852c-4e28-bc46-630d4dfa0986 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:39.205134 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-57278874-852c-4e28-bc46-630d4dfa0986 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:39.219260 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57278874-852c-4e28-bc46-630d4dfa0986 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:39.304411 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df41d159-debc-4c83-adfe-8ff396a4bcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:39.305966 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41d159-debc-4c83-adfe-8ff396a4bcc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:39.306482 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41d159-debc-4c83-adfe-8ff396a4bcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:39.307126 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41d159-debc-4c83-adfe-8ff396a4bcc2 admin admin] PATH_INFO: `/v3/projects/64c43c586a7b4e7eab7c4a0bd59e5eba` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:39.308896 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df41d159-debc-4c83-adfe-8ff396a4bcc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64c43c586a7b4e7eab7c4a0bd59e5eba)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:39.311999 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df41d159-debc-4c83-adfe-8ff396a4bcc2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:39.466014 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-df41d159-debc-4c83-adfe-8ff396a4bcc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64c43c586a7b4e7eab7c4a0bd59e5eba', 'request_id': 'req-df41d159-debc-4c83-adfe-8ff396a4bcc2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:39.473082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-df41d159-debc-4c83-adfe-8ff396a4bcc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64c43c586a7b4e7eab7c4a0bd59e5eba', 'request_id': 'req-df41d159-debc-4c83-adfe-8ff396a4bcc2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:39.479909 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-df41d159-debc-4c83-adfe-8ff396a4bcc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64c43c586a7b4e7eab7c4a0bd59e5eba', 'request_id': 'req-df41d159-debc-4c83-adfe-8ff396a4bcc2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:39.626932 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-df41d159-debc-4c83-adfe-8ff396a4bcc2 admin admin] The token cache is being invalidate because project 64c43c586a7b4e7eab7c4a0bd59e5eba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:34:39.627413 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-df41d159-debc-4c83-adfe-8ff396a4bcc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:39.628371 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-df41d159-debc-4c83-adfe-8ff396a4bcc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:39.630684 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 890/1785] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:34:39 2022] DELETE /identity/v3/projects/64c43c586a7b4e7eab7c4a0bd59e5eba => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:39.636619 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45623483-ccc0-4ca5-82ae-72a619023460 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:39.639441 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-45623483-ccc0-4ca5-82ae-72a619023460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:39.663689 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45623483-ccc0-4ca5-82ae-72a619023460 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:39.753026 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c130877-41da-4b4d-a3c7-84152385cda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:39.754468 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c130877-41da-4b4d-a3c7-84152385cda4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:39.754939 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c130877-41da-4b4d-a3c7-84152385cda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:39.755238 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c130877-41da-4b4d-a3c7-84152385cda4 admin admin] PATH_INFO: `/v3/users/110afb9f6c39468aa31830f072ed5c91` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:39.757670 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c130877-41da-4b4d-a3c7-84152385cda4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=110afb9f6c39468aa31830f072ed5c91)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:39.760674 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c130877-41da-4b4d-a3c7-84152385cda4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:40.017143 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df41d159-debc-4c83-adfe-8ff396a4bcc2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:40.018610 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-df41d159-debc-4c83-adfe-8ff396a4bcc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:40.034053 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df41d159-debc-4c83-adfe-8ff396a4bcc2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:40.042495 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ec24f-18f2-4ddd-b7c3-6c6297dee96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:40.044151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ec24f-18f2-4ddd-b7c3-6c6297dee96b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:40.045215 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ec24f-18f2-4ddd-b7c3-6c6297dee96b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:40.045983 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ec24f-18f2-4ddd-b7c3-6c6297dee96b admin admin] PATH_INFO: `/v3/projects/cf9a16b12c734de8b2233e3f2658a80d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:40.048061 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ec24f-18f2-4ddd-b7c3-6c6297dee96b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf9a16b12c734de8b2233e3f2658a80d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:40.051826 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ec24f-18f2-4ddd-b7c3-6c6297dee96b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:40.223794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3c130877-41da-4b4d-a3c7-84152385cda4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '110afb9f6c39468aa31830f072ed5c91'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:40.224987 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3c130877-41da-4b4d-a3c7-84152385cda4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '110afb9f6c39468aa31830f072ed5c91'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:40.236485 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3c130877-41da-4b4d-a3c7-84152385cda4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '110afb9f6c39468aa31830f072ed5c91'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:40.243508 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-699ec24f-18f2-4ddd-b7c3-6c6297dee96b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf9a16b12c734de8b2233e3f2658a80d', 'request_id': 'req-699ec24f-18f2-4ddd-b7c3-6c6297dee96b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:40.254297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-699ec24f-18f2-4ddd-b7c3-6c6297dee96b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf9a16b12c734de8b2233e3f2658a80d', 'request_id': 'req-699ec24f-18f2-4ddd-b7c3-6c6297dee96b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:40.261275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-699ec24f-18f2-4ddd-b7c3-6c6297dee96b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf9a16b12c734de8b2233e3f2658a80d', 'request_id': 'req-699ec24f-18f2-4ddd-b7c3-6c6297dee96b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:40.362890 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3c130877-41da-4b4d-a3c7-84152385cda4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '110afb9f6c39468aa31830f072ed5c91'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:40.363586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3c130877-41da-4b4d-a3c7-84152385cda4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '110afb9f6c39468aa31830f072ed5c91'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:40.386935 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 896/1786] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:34:39 2022] DELETE /identity/v3/users/110afb9f6c39468aa31830f072ed5c91 => generated 0 bytes in 752 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:40.432489 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-699ec24f-18f2-4ddd-b7c3-6c6297dee96b admin admin] The token cache is being invalidate because project cf9a16b12c734de8b2233e3f2658a80d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:34:40.432884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-699ec24f-18f2-4ddd-b7c3-6c6297dee96b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:40.433795 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-699ec24f-18f2-4ddd-b7c3-6c6297dee96b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:40.436071 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 891/1787] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:34:40 2022] DELETE /identity/v3/projects/cf9a16b12c734de8b2233e3f2658a80d => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:34:40.442901 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f09b2-9583-4f80-b146-88451183a381 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:40.443196 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f09b2-9583-4f80-b146-88451183a381 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:40.443503 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f09b2-9583-4f80-b146-88451183a381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:40.483983 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9b1f09b2-9583-4f80-b146-88451183a381 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:40.485288 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9b1f09b2-9583-4f80-b146-88451183a381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:40.548049 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:40.609252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 897/1788] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:40.617318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be00001-f66c-4a3c-b0a1-881ef420c4a2 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:40.617617 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be00001-f66c-4a3c-b0a1-881ef420c4a2 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:40.617881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be00001-f66c-4a3c-b0a1-881ef420c4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:40.643744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0be00001-f66c-4a3c-b0a1-881ef420c4a2 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:40.644430 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0be00001-f66c-4a3c-b0a1-881ef420c4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:40.659076 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 892/1789] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:34:40.666408 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f09b2-9583-4f80-b146-88451183a381 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:40.683286 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:40.689124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f09b2-9583-4f80-b146-88451183a381 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:40.696480 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b380b27-0309-4b98-9bed-ccf13afd5e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:40.698228 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b380b27-0309-4b98-9bed-ccf13afd5e7f admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:40.698711 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b380b27-0309-4b98-9bed-ccf13afd5e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:40.700201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b380b27-0309-4b98-9bed-ccf13afd5e7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:40.703753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b380b27-0309-4b98-9bed-ccf13afd5e7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:34:40.704539 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b380b27-0309-4b98-9bed-ccf13afd5e7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:40.708323 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b380b27-0309-4b98-9bed-ccf13afd5e7f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:40.728922 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 898/1790] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:34:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:34:40.730222 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be00001-f66c-4a3c-b0a1-881ef420c4a2 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:40.755839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be00001-f66c-4a3c-b0a1-881ef420c4a2 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:40.764013 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e2316-ce25-4a9a-9ec9-f1a427a8607a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:40.764791 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e2316-ce25-4a9a-9ec9-f1a427a8607a admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:40.765264 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e2316-ce25-4a9a-9ec9-f1a427a8607a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:40.765537 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e2316-ce25-4a9a-9ec9-f1a427a8607a admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:40.767239 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e2316-ce25-4a9a-9ec9-f1a427a8607a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:40.770241 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e2316-ce25-4a9a-9ec9-f1a427a8607a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:40.975251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 893/1791] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:34:40 2022] POST /identity/v3/projects => generated 381 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:34:40.976925 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b380b27-0309-4b98-9bed-ccf13afd5e7f admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:40.995256 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b380b27-0309-4b98-9bed-ccf13afd5e7f admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:41.012877 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e2316-ce25-4a9a-9ec9-f1a427a8607a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:41.016101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8d1e2316-ce25-4a9a-9ec9-f1a427a8607a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:41.105015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30eeb35-91b5-4a45-8819-92b0825aa71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:41.106159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30eeb35-91b5-4a45-8819-92b0825aa71e admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:41.106835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30eeb35-91b5-4a45-8819-92b0825aa71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:41.107264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30eeb35-91b5-4a45-8819-92b0825aa71e admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:41.108288 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30eeb35-91b5-4a45-8819-92b0825aa71e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:41.110792 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30eeb35-91b5-4a45-8819-92b0825aa71e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:41.112317 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e2316-ce25-4a9a-9ec9-f1a427a8607a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:41.117748 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efbd852-807d-474d-a9fb-6b5e2280b47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:41.118629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efbd852-807d-474d-a9fb-6b5e2280b47a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:41.118907 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efbd852-807d-474d-a9fb-6b5e2280b47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:41.119201 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efbd852-807d-474d-a9fb-6b5e2280b47a admin admin] PATH_INFO: `/v3/projects/1cf8f499cd9f49a49f82b06c0b14c436` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:41.120633 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efbd852-807d-474d-a9fb-6b5e2280b47a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cf8f499cd9f49a49f82b06c0b14c436)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:41.122555 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efbd852-807d-474d-a9fb-6b5e2280b47a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:41.225353 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 899/1792] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:34:40 2022] POST /identity/v3/users => generated 291 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:41.229455 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30eeb35-91b5-4a45-8819-92b0825aa71e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:41.239937 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30eeb35-91b5-4a45-8819-92b0825aa71e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:41.244590 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4456f-1ad7-4344-862c-4ca2cdb0052b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:41.245564 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4456f-1ad7-4344-862c-4ca2cdb0052b admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:41.245856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4456f-1ad7-4344-862c-4ca2cdb0052b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:41.246163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4456f-1ad7-4344-862c-4ca2cdb0052b admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:41.246920 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf4456f-1ad7-4344-862c-4ca2cdb0052b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:41.248910 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf4456f-1ad7-4344-862c-4ca2cdb0052b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:41.259374 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 900/1793] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:34:41.263956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4456f-1ad7-4344-862c-4ca2cdb0052b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:41.266255 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9efbd852-807d-474d-a9fb-6b5e2280b47a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cf8f499cd9f49a49f82b06c0b14c436', 'request_id': 'req-9efbd852-807d-474d-a9fb-6b5e2280b47a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:41.272999 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9efbd852-807d-474d-a9fb-6b5e2280b47a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cf8f499cd9f49a49f82b06c0b14c436', 'request_id': 'req-9efbd852-807d-474d-a9fb-6b5e2280b47a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:41.274681 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4456f-1ad7-4344-862c-4ca2cdb0052b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:41.278725 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9efbd852-807d-474d-a9fb-6b5e2280b47a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cf8f499cd9f49a49f82b06c0b14c436', 'request_id': 'req-9efbd852-807d-474d-a9fb-6b5e2280b47a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:41.334698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c983695-3c5b-494f-a9fa-9d790bf4c0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:41.335370 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c983695-3c5b-494f-a9fa-9d790bf4c0c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:41.335679 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c983695-3c5b-494f-a9fa-9d790bf4c0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:41.335951 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c983695-3c5b-494f-a9fa-9d790bf4c0c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:41.336675 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c983695-3c5b-494f-a9fa-9d790bf4c0c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:41.338287 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c983695-3c5b-494f-a9fa-9d790bf4c0c8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:41.350617 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 901/1794] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:34:41.355375 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c983695-3c5b-494f-a9fa-9d790bf4c0c8 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:41.365738 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c983695-3c5b-494f-a9fa-9d790bf4c0c8 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:41.370768 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b643c-9753-436e-b433-c3dfeccd7688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:41.371729 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b643c-9753-436e-b433-c3dfeccd7688 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:41.372057 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b643c-9753-436e-b433-c3dfeccd7688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:41.372323 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b643c-9753-436e-b433-c3dfeccd7688 admin admin] PATH_INFO: `/v3/projects/bb444f73f7c64b4fadd6f2d3c89eb3d1/users/88125f681fe44be9a5f59f1a47161054/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:41.391107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9efbd852-807d-474d-a9fb-6b5e2280b47a admin admin] The token cache is being invalidate because project 1cf8f499cd9f49a49f82b06c0b14c436 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:34:41.391402 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9efbd852-807d-474d-a9fb-6b5e2280b47a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:41.392299 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9efbd852-807d-474d-a9fb-6b5e2280b47a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:41.393322 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b643c-9753-436e-b433-c3dfeccd7688 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb444f73f7c64b4fadd6f2d3c89eb3d1, user_id=88125f681fe44be9a5f59f1a47161054, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:41.395668 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 894/1795] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:34:41 2022] DELETE /identity/v3/projects/1cf8f499cd9f49a49f82b06c0b14c436 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:41.402883 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f6d48-4348-4c7a-961a-bb16568b77dd None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:41.402990 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b643c-9753-436e-b433-c3dfeccd7688 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:41.403081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f6d48-4348-4c7a-961a-bb16568b77dd None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:41.403345 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f6d48-4348-4c7a-961a-bb16568b77dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:41.423497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-253f6d48-4348-4c7a-961a-bb16568b77dd None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:41.424766 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-253f6d48-4348-4c7a-961a-bb16568b77dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:41.433789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 895/1796] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:34:41.442631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb097de-42d5-43c0-b140-ce993f041321 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:41.443083 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb097de-42d5-43c0-b140-ce993f041321 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:41.443371 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb097de-42d5-43c0-b140-ce993f041321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:41.463369 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fdb097de-42d5-43c0-b140-ce993f041321 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:41.467515 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fdb097de-42d5-43c0-b140-ce993f041321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:41.478827 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 896/1797] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:34:41.483888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb097de-42d5-43c0-b140-ce993f041321 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:41.493791 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb097de-42d5-43c0-b140-ce993f041321 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:41.500009 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9285825-1b8a-47c0-b7f6-2ad39320ff1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:41.500691 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9285825-1b8a-47c0-b7f6-2ad39320ff1a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:41.506119 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9285825-1b8a-47c0-b7f6-2ad39320ff1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:41.506119 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9285825-1b8a-47c0-b7f6-2ad39320ff1a admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:41.506119 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9285825-1b8a-47c0-b7f6-2ad39320ff1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:34:41.506119 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9285825-1b8a-47c0-b7f6-2ad39320ff1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:41.506119 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9285825-1b8a-47c0-b7f6-2ad39320ff1a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:41.520443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 897/1798] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:34:41.520609 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 902/1799] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:41 2022] PUT /identity/v3/projects/bb444f73f7c64b4fadd6f2d3c89eb3d1/users/88125f681fe44be9a5f59f1a47161054/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:34:41.522287 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9285825-1b8a-47c0-b7f6-2ad39320ff1a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:41.530081 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e7f6a-1438-4c87-a70b-a41c8723cc11 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:41.530420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e7f6a-1438-4c87-a70b-a41c8723cc11 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:41.530708 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222e7f6a-1438-4c87-a70b-a41c8723cc11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:41.532915 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9285825-1b8a-47c0-b7f6-2ad39320ff1a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:41.562115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-222e7f6a-1438-4c87-a70b-a41c8723cc11 None None] MFA Rules not processed for user `88125f681fe44be9a5f59f1a47161054`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:41.563570 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-222e7f6a-1438-4c87-a70b-a41c8723cc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:41.597375 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb4f11-64c2-4992-9ffb-7cff5d5466d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:41.598062 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb4f11-64c2-4992-9ffb-7cff5d5466d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:41.598367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb4f11-64c2-4992-9ffb-7cff5d5466d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:41.598645 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb4f11-64c2-4992-9ffb-7cff5d5466d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:41.599378 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb4f11-64c2-4992-9ffb-7cff5d5466d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:41.601356 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb4f11-64c2-4992-9ffb-7cff5d5466d7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:41.647193 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 903/1800] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:34:41.651791 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e7f6a-1438-4c87-a70b-a41c8723cc11 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:41.653082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-222e7f6a-1438-4c87-a70b-a41c8723cc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:41.666612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222e7f6a-1438-4c87-a70b-a41c8723cc11 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:41.671691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039bf24d-4528-44b7-85cc-afdf63e736a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:41.672360 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bf24d-4528-44b7-85cc-afdf63e736a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:41.672647 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bf24d-4528-44b7-85cc-afdf63e736a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:41.672928 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039bf24d-4528-44b7-85cc-afdf63e736a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:41.673637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039bf24d-4528-44b7-85cc-afdf63e736a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:41.675685 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039bf24d-4528-44b7-85cc-afdf63e736a4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:41.704291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 898/1801] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:34:41 2022] POST /identity/v3/projects => generated 381 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:41.708514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb4f11-64c2-4992-9ffb-7cff5d5466d7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:41.791427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 904/1802] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:34:41 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:34:41.799172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039bf24d-4528-44b7-85cc-afdf63e736a4 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:41.816144 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb4f11-64c2-4992-9ffb-7cff5d5466d7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:41.909968 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039bf24d-4528-44b7-85cc-afdf63e736a4 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:41.918108 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d6ade-7da5-4123-88ba-6e85abe2f92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:41.918814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d6ade-7da5-4123-88ba-6e85abe2f92a admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:41.919248 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d6ade-7da5-4123-88ba-6e85abe2f92a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:41.919658 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d6ade-7da5-4123-88ba-6e85abe2f92a admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:41.922702 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d6ade-7da5-4123-88ba-6e85abe2f92a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:41.925430 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d6ade-7da5-4123-88ba-6e85abe2f92a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:41.945939 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7df0a0-9cae-4e19-95cf-214360e5b15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:41.965548 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7df0a0-9cae-4e19-95cf-214360e5b15f admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:41.968791 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7df0a0-9cae-4e19-95cf-214360e5b15f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:41.969247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7df0a0-9cae-4e19-95cf-214360e5b15f admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:41.974330 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7df0a0-9cae-4e19-95cf-214360e5b15f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:41.978411 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7df0a0-9cae-4e19-95cf-214360e5b15f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:42.029079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 905/1803] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:34:41 2022] POST /identity/v3/users => generated 296 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:42.034092 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d6ade-7da5-4123-88ba-6e85abe2f92a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:42.060280 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d6ade-7da5-4123-88ba-6e85abe2f92a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:42.070713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a337cc-2e03-41ca-9a93-134e29c32dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:42.071415 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a337cc-2e03-41ca-9a93-134e29c32dbb admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:42.071944 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a337cc-2e03-41ca-9a93-134e29c32dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:42.072050 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a337cc-2e03-41ca-9a93-134e29c32dbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:42.072747 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a337cc-2e03-41ca-9a93-134e29c32dbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:42.074450 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a337cc-2e03-41ca-9a93-134e29c32dbb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:42.090604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 906/1804] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:34:42.092231 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a337cc-2e03-41ca-9a93-134e29c32dbb admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:42.110933 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a337cc-2e03-41ca-9a93-134e29c32dbb admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:42.121852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762ad11-1462-42da-80cb-1c6b182c7d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:42.122424 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762ad11-1462-42da-80cb-1c6b182c7d12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:42.122929 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762ad11-1462-42da-80cb-1c6b182c7d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:42.122929 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762ad11-1462-42da-80cb-1c6b182c7d12 admin admin] PATH_INFO: `/v3/projects/d53f6811380a4832af1e2c436d97fbf3/users/35e3ebe1ddc546bf8b10304e6417d4b5/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:42.143053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762ad11-1462-42da-80cb-1c6b182c7d12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d53f6811380a4832af1e2c436d97fbf3, user_id=35e3ebe1ddc546bf8b10304e6417d4b5, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:42.146019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762ad11-1462-42da-80cb-1c6b182c7d12 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:42.188402 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 899/1805] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:34:41 2022] POST /identity/v3/users => generated 291 bytes in 480 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:34:42.192778 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7df0a0-9cae-4e19-95cf-214360e5b15f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:42.204753 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7df0a0-9cae-4e19-95cf-214360e5b15f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:42.209929 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef585bf9-006a-4756-aa1f-70c71da331ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:42.210868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef585bf9-006a-4756-aa1f-70c71da331ab admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:42.211200 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef585bf9-006a-4756-aa1f-70c71da331ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:42.211588 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef585bf9-006a-4756-aa1f-70c71da331ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:42.212405 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef585bf9-006a-4756-aa1f-70c71da331ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:42.214355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef585bf9-006a-4756-aa1f-70c71da331ab admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:42.226866 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 900/1806] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:34:42.230729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef585bf9-006a-4756-aa1f-70c71da331ab admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:42.244426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef585bf9-006a-4756-aa1f-70c71da331ab admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:42.250033 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1573cea6-8322-494d-b287-e4ec7c10808a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:42.250796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573cea6-8322-494d-b287-e4ec7c10808a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:42.251146 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573cea6-8322-494d-b287-e4ec7c10808a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:42.251517 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1573cea6-8322-494d-b287-e4ec7c10808a admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:42.252323 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1573cea6-8322-494d-b287-e4ec7c10808a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:42.254128 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1573cea6-8322-494d-b287-e4ec7c10808a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:42.266336 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 901/1807] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:34:42.270713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1573cea6-8322-494d-b287-e4ec7c10808a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:42.278433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 907/1808] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:42 2022] PUT /identity/v3/projects/d53f6811380a4832af1e2c436d97fbf3/users/35e3ebe1ddc546bf8b10304e6417d4b5/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:42.282442 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1573cea6-8322-494d-b287-e4ec7c10808a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:42.285322 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762ad11-1462-42da-80cb-1c6b182c7d12 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:42.355120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762ad11-1462-42da-80cb-1c6b182c7d12 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:42.357012 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cfd81d-bcb5-457e-a9be-ad6c244f155f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:42.358852 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfd81d-bcb5-457e-a9be-ad6c244f155f admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:42.358852 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfd81d-bcb5-457e-a9be-ad6c244f155f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:42.359140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cfd81d-bcb5-457e-a9be-ad6c244f155f admin admin] PATH_INFO: `/v3/projects/d353005498fc443dbb6026b2599347ea/users/b0775f2a2b924af28cad657ebf3e6f78/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:42.368302 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15853d4e-b605-4f7e-9d4b-415b3d734f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:42.369484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15853d4e-b605-4f7e-9d4b-415b3d734f55 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:42.369995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15853d4e-b605-4f7e-9d4b-415b3d734f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:42.370555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15853d4e-b605-4f7e-9d4b-415b3d734f55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:42.371310 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15853d4e-b605-4f7e-9d4b-415b3d734f55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:42.373233 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15853d4e-b605-4f7e-9d4b-415b3d734f55 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:42.378687 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cfd81d-bcb5-457e-a9be-ad6c244f155f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d353005498fc443dbb6026b2599347ea, user_id=b0775f2a2b924af28cad657ebf3e6f78, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:42.382133 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cfd81d-bcb5-457e-a9be-ad6c244f155f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:42.385772 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 908/1809] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:42.390154 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15853d4e-b605-4f7e-9d4b-415b3d734f55 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:42.401007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15853d4e-b605-4f7e-9d4b-415b3d734f55 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:42.406455 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ecd142-2cbf-40b3-8d44-7aca5e9de6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:42.407404 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecd142-2cbf-40b3-8d44-7aca5e9de6e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:42.407786 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecd142-2cbf-40b3-8d44-7aca5e9de6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:42.408118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ecd142-2cbf-40b3-8d44-7aca5e9de6e7 admin admin] PATH_INFO: `/v3/projects/d53f6811380a4832af1e2c436d97fbf3/users/35e3ebe1ddc546bf8b10304e6417d4b5/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:42.410597 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ecd142-2cbf-40b3-8d44-7aca5e9de6e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d53f6811380a4832af1e2c436d97fbf3, user_id=35e3ebe1ddc546bf8b10304e6417d4b5, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:42.413371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ecd142-2cbf-40b3-8d44-7aca5e9de6e7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:42.425110 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 909/1810] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:42 2022] PUT /identity/v3/projects/d53f6811380a4832af1e2c436d97fbf3/users/35e3ebe1ddc546bf8b10304e6417d4b5/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:34:42.435007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935981eb-3583-4a18-8b80-1186d947dce1 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:42.435299 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935981eb-3583-4a18-8b80-1186d947dce1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:42.435644 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935981eb-3583-4a18-8b80-1186d947dce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:42.466723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-935981eb-3583-4a18-8b80-1186d947dce1 None None] MFA Rules not processed for user `35e3ebe1ddc546bf8b10304e6417d4b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:42.469061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-935981eb-3583-4a18-8b80-1186d947dce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:42.534412 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 902/1811] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:42 2022] PUT /identity/v3/projects/d353005498fc443dbb6026b2599347ea/users/b0775f2a2b924af28cad657ebf3e6f78/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:34:42.544733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fb0f1-4e5a-4a77-b49f-94b47fc76141 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:42.545154 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fb0f1-4e5a-4a77-b49f-94b47fc76141 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:42.545518 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fb0f1-4e5a-4a77-b49f-94b47fc76141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:42.561100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 910/1812] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:42 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:34:42.574024 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935981eb-3583-4a18-8b80-1186d947dce1 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:42.575680 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-935981eb-3583-4a18-8b80-1186d947dce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:42.591399 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-255fb0f1-4e5a-4a77-b49f-94b47fc76141 None None] MFA Rules not processed for user `b0775f2a2b924af28cad657ebf3e6f78`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:42.593596 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-255fb0f1-4e5a-4a77-b49f-94b47fc76141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:42.667450 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935981eb-3583-4a18-8b80-1186d947dce1 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:42.697848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 903/1813] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:42.703028 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255fb0f1-4e5a-4a77-b49f-94b47fc76141 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:42.761637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7155c-a943-4389-b2e8-174bde5427b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:42.762681 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7155c-a943-4389-b2e8-174bde5427b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:42.763113 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7155c-a943-4389-b2e8-174bde5427b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:42.763568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7155c-a943-4389-b2e8-174bde5427b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:42.827598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255fb0f1-4e5a-4a77-b49f-94b47fc76141 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:42.833879 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2992d537-dc83-433a-aadf-0eef8b49cb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:42.834885 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992d537-dc83-433a-aadf-0eef8b49cb3e admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:42.835244 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992d537-dc83-433a-aadf-0eef8b49cb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:42.835635 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992d537-dc83-433a-aadf-0eef8b49cb3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:42.838345 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2992d537-dc83-433a-aadf-0eef8b49cb3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:42.841249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2992d537-dc83-433a-aadf-0eef8b49cb3e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:42.915640 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7155c-a943-4389-b2e8-174bde5427b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:42.918543 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7155c-a943-4389-b2e8-174bde5427b8 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:42.928723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 911/1814] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:42 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:34:43.043729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 904/1815] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:34:42 2022] POST /identity/v3/projects => generated 381 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:43.048742 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7155c-a943-4389-b2e8-174bde5427b8 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:43.059682 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7155c-a943-4389-b2e8-174bde5427b8 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:43.150913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ba1e7-0c4d-4ede-bc3b-9948ef221c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:43.151684 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ba1e7-0c4d-4ede-bc3b-9948ef221c48 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:43.151975 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ba1e7-0c4d-4ede-bc3b-9948ef221c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:43.152258 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ba1e7-0c4d-4ede-bc3b-9948ef221c48 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:43.152971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ba1e7-0c4d-4ede-bc3b-9948ef221c48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:43.155597 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ba1e7-0c4d-4ede-bc3b-9948ef221c48 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:43.292479 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 912/1816] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:34:43 2022] POST /identity/v3/users => generated 297 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:43.293675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2992d537-dc83-433a-aadf-0eef8b49cb3e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:43.308718 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2992d537-dc83-433a-aadf-0eef8b49cb3e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:43.316818 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92fbb1-f6ea-467d-bdb3-4cbb75c64e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:43.317474 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92fbb1-f6ea-467d-bdb3-4cbb75c64e7a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:43.317756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92fbb1-f6ea-467d-bdb3-4cbb75c64e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:43.318029 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92fbb1-f6ea-467d-bdb3-4cbb75c64e7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:43.318761 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f92fbb1-f6ea-467d-bdb3-4cbb75c64e7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:43.320628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f92fbb1-f6ea-467d-bdb3-4cbb75c64e7a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:43.333221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 905/1817] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:43.337310 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ba1e7-0c4d-4ede-bc3b-9948ef221c48 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:43.348614 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ba1e7-0c4d-4ede-bc3b-9948ef221c48 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:43.356228 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea06cd7-4a7e-44e5-9d6d-fa31ce5aabb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:43.356228 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea06cd7-4a7e-44e5-9d6d-fa31ce5aabb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:43.356228 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea06cd7-4a7e-44e5-9d6d-fa31ce5aabb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:43.356228 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea06cd7-4a7e-44e5-9d6d-fa31ce5aabb6 admin admin] PATH_INFO: `/v3/projects/fc0c828632bf46949b79badfb31e0b7e/users/8a76c4316c64403b93159e5b87d994a0/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:43.384386 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea06cd7-4a7e-44e5-9d6d-fa31ce5aabb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc0c828632bf46949b79badfb31e0b7e, user_id=8a76c4316c64403b93159e5b87d994a0, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:43.385735 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea06cd7-4a7e-44e5-9d6d-fa31ce5aabb6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:43.468924 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:43.482125 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 913/1818] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:43 2022] PUT /identity/v3/projects/fc0c828632bf46949b79badfb31e0b7e/users/8a76c4316c64403b93159e5b87d994a0/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:34:43.484644 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92fbb1-f6ea-467d-bdb3-4cbb75c64e7a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:43.502432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92fbb1-f6ea-467d-bdb3-4cbb75c64e7a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:43.603129 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec9ed1-404d-44c8-9ff1-775e94e744a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:43.603765 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec9ed1-404d-44c8-9ff1-775e94e744a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:43.604805 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec9ed1-404d-44c8-9ff1-775e94e744a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:43.605154 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec9ed1-404d-44c8-9ff1-775e94e744a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:43.605903 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec9ed1-404d-44c8-9ff1-775e94e744a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:43.607577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec9ed1-404d-44c8-9ff1-775e94e744a7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:43.620611 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 906/1819] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:34:43.628765 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea06cd7-4a7e-44e5-9d6d-fa31ce5aabb6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:43.641387 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea06cd7-4a7e-44e5-9d6d-fa31ce5aabb6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:43.650846 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22431a94-cb00-4ba4-a349-5afd8aa90179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:43.651948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22431a94-cb00-4ba4-a349-5afd8aa90179 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:43.652569 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22431a94-cb00-4ba4-a349-5afd8aa90179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:43.652870 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22431a94-cb00-4ba4-a349-5afd8aa90179 admin admin] PATH_INFO: `/v3/projects/fc0c828632bf46949b79badfb31e0b7e/users/8a76c4316c64403b93159e5b87d994a0/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:43.657410 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22431a94-cb00-4ba4-a349-5afd8aa90179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc0c828632bf46949b79badfb31e0b7e, user_id=8a76c4316c64403b93159e5b87d994a0, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:43.660173 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22431a94-cb00-4ba4-a349-5afd8aa90179 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:43.675065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec9ed1-404d-44c8-9ff1-775e94e744a7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:43.687181 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 914/1820] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:43 2022] PUT /identity/v3/projects/fc0c828632bf46949b79badfb31e0b7e/users/8a76c4316c64403b93159e5b87d994a0/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:43.688422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec9ed1-404d-44c8-9ff1-775e94e744a7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:43.692693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4656c-a748-40d2-8527-479c7418fd3f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:43.692982 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4656c-a748-40d2-8527-479c7418fd3f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:43.693260 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4656c-a748-40d2-8527-479c7418fd3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:43.732650 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-bdc4656c-a748-40d2-8527-479c7418fd3f None None] MFA Rules not processed for user `8a76c4316c64403b93159e5b87d994a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:43.735138 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bdc4656c-a748-40d2-8527-479c7418fd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:43.791613 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e4a4a-8f5c-4e63-b434-ec688159139c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:43.792655 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e4a4a-8f5c-4e63-b434-ec688159139c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:43.792943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e4a4a-8f5c-4e63-b434-ec688159139c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:43.797203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e4a4a-8f5c-4e63-b434-ec688159139c admin admin] PATH_INFO: `/v3/users/88125f681fe44be9a5f59f1a47161054` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:43.800474 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e4a4a-8f5c-4e63-b434-ec688159139c admin admin] RBAC: Authorizing `identity:delete_user(user_id=88125f681fe44be9a5f59f1a47161054)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:43.804250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e4a4a-8f5c-4e63-b434-ec688159139c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:43.848706 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 915/1821] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:43 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:43.864142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4656c-a748-40d2-8527-479c7418fd3f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:43.939744 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4656c-a748-40d2-8527-479c7418fd3f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:43.944144 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6dcab2-5ff4-448d-9a56-cfb38139217b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:43.945101 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6dcab2-5ff4-448d-9a56-cfb38139217b service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:43.945387 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6dcab2-5ff4-448d-9a56-cfb38139217b service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:43.945849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6dcab2-5ff4-448d-9a56-cfb38139217b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:43.958464 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6dcab2-5ff4-448d-9a56-cfb38139217b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:43.960301 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6dcab2-5ff4-448d-9a56-cfb38139217b service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:43.976403 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 916/1822] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:43 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:34:44.120041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c71e4a4a-8f5c-4e63-b434-ec688159139c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88125f681fe44be9a5f59f1a47161054'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:44.123463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c71e4a4a-8f5c-4e63-b434-ec688159139c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88125f681fe44be9a5f59f1a47161054'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:44.198193 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c71e4a4a-8f5c-4e63-b434-ec688159139c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88125f681fe44be9a5f59f1a47161054'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:44.210558 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c71e4a4a-8f5c-4e63-b434-ec688159139c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88125f681fe44be9a5f59f1a47161054'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:44.213450 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c71e4a4a-8f5c-4e63-b434-ec688159139c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88125f681fe44be9a5f59f1a47161054'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:44.259048 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 907/1823] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:34:43 2022] DELETE /identity/v3/users/88125f681fe44be9a5f59f1a47161054 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:34:44.265375 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79b153-df04-4f73-b39f-ae22ef722399 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:44.265375 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79b153-df04-4f73-b39f-ae22ef722399 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:44.265375 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79b153-df04-4f73-b39f-ae22ef722399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:44.289956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ec79b153-df04-4f73-b39f-ae22ef722399 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:44.289956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ec79b153-df04-4f73-b39f-ae22ef722399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:44.377420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 917/1824] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:44.391067 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e4a4a-8f5c-4e63-b434-ec688159139c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:44.393962 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c71e4a4a-8f5c-4e63-b434-ec688159139c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:44.517933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e4a4a-8f5c-4e63-b434-ec688159139c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:44.534193 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65097bf6-90b0-4503-9118-58a7d9cf1830 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:44.534193 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65097bf6-90b0-4503-9118-58a7d9cf1830 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:44.534193 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65097bf6-90b0-4503-9118-58a7d9cf1830 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:44.534193 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65097bf6-90b0-4503-9118-58a7d9cf1830 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:44.539167 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65097bf6-90b0-4503-9118-58a7d9cf1830 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:44.540687 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65097bf6-90b0-4503-9118-58a7d9cf1830 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:44.552092 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 908/1825] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:34:44.594772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:45.575248 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec79b153-df04-4f73-b39f-ae22ef722399 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:45.577416 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ec79b153-df04-4f73-b39f-ae22ef722399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:45.590443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec79b153-df04-4f73-b39f-ae22ef722399 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:45.607898 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e951cb-c0ac-43ce-b98e-2d1915d8e482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:45.609198 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e951cb-c0ac-43ce-b98e-2d1915d8e482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:45.609502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e951cb-c0ac-43ce-b98e-2d1915d8e482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:45.609793 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e951cb-c0ac-43ce-b98e-2d1915d8e482 admin admin] PATH_INFO: `/v3/projects/bb444f73f7c64b4fadd6f2d3c89eb3d1` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:45.612028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e951cb-c0ac-43ce-b98e-2d1915d8e482 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb444f73f7c64b4fadd6f2d3c89eb3d1)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:45.614048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e951cb-c0ac-43ce-b98e-2d1915d8e482 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:45.844903 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65097bf6-90b0-4503-9118-58a7d9cf1830 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:45.846777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-65097bf6-90b0-4503-9118-58a7d9cf1830 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:45.871821 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-82e951cb-c0ac-43ce-b98e-2d1915d8e482 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb444f73f7c64b4fadd6f2d3c89eb3d1', 'request_id': 'req-82e951cb-c0ac-43ce-b98e-2d1915d8e482'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:45.979326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65097bf6-90b0-4503-9118-58a7d9cf1830 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:46.062436 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-82e951cb-c0ac-43ce-b98e-2d1915d8e482 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb444f73f7c64b4fadd6f2d3c89eb3d1', 'request_id': 'req-82e951cb-c0ac-43ce-b98e-2d1915d8e482'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:46.073789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-82e951cb-c0ac-43ce-b98e-2d1915d8e482 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb444f73f7c64b4fadd6f2d3c89eb3d1', 'request_id': 'req-82e951cb-c0ac-43ce-b98e-2d1915d8e482'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:46.106367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-82e951cb-c0ac-43ce-b98e-2d1915d8e482 admin admin] The token cache is being invalidate because project bb444f73f7c64b4fadd6f2d3c89eb3d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:34:46.108784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-82e951cb-c0ac-43ce-b98e-2d1915d8e482 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:46.111702 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-82e951cb-c0ac-43ce-b98e-2d1915d8e482 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:46.121645 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 918/1826] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:34:45 2022] DELETE /identity/v3/projects/bb444f73f7c64b4fadd6f2d3c89eb3d1 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:46.122358 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e951cb-c0ac-43ce-b98e-2d1915d8e482 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:46.140998 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-82e951cb-c0ac-43ce-b98e-2d1915d8e482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:46.154294 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1299383-2f3d-4417-a089-d178f53174d6 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a76c4316c64403b93159e5b87d994a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc0c828632bf46949b79badfb31e0b7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:46.156098 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1299383-2f3d-4417-a089-d178f53174d6 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:46.157164 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1299383-2f3d-4417-a089-d178f53174d6 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:46.158492 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1299383-2f3d-4417-a089-d178f53174d6 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:46.160932 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1299383-2f3d-4417-a089-d178f53174d6 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:34:46.161544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1299383-2f3d-4417-a089-d178f53174d6 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:46.163680 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1299383-2f3d-4417-a089-d178f53174d6 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:46.188367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 909/1827] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:34:46.193828 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1299383-2f3d-4417-a089-d178f53174d6 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:46.200166 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a1299383-2f3d-4417-a089-d178f53174d6 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:46.235770 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1299383-2f3d-4417-a089-d178f53174d6 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:46.251159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e951cb-c0ac-43ce-b98e-2d1915d8e482 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:46.251778 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69c0cd-eb4f-4f9a-a26a-88e71f5dafbb tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a76c4316c64403b93159e5b87d994a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc0c828632bf46949b79badfb31e0b7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:46.254203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69c0cd-eb4f-4f9a-a26a-88e71f5dafbb tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:46.256650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69c0cd-eb4f-4f9a-a26a-88e71f5dafbb tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:46.256650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69c0cd-eb4f-4f9a-a26a-88e71f5dafbb tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:46.257280 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d131a1a-1491-488c-995b-be9fd76f25ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:46.259017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d131a1a-1491-488c-995b-be9fd76f25ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:46.259381 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d131a1a-1491-488c-995b-be9fd76f25ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:46.259876 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff69c0cd-eb4f-4f9a-a26a-88e71f5dafbb tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:46.260009 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d131a1a-1491-488c-995b-be9fd76f25ce admin admin] PATH_INFO: `/v3/users/35e3ebe1ddc546bf8b10304e6417d4b5` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:46.261582 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d131a1a-1491-488c-995b-be9fd76f25ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=35e3ebe1ddc546bf8b10304e6417d4b5)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:46.265635 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d131a1a-1491-488c-995b-be9fd76f25ce admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:46.266700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff69c0cd-eb4f-4f9a-a26a-88e71f5dafbb tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:46.431011 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 910/1828] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:34:46 2022] POST /identity/v3/projects => generated 354 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:46.434026 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69c0cd-eb4f-4f9a-a26a-88e71f5dafbb tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:46.549553 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69c0cd-eb4f-4f9a-a26a-88e71f5dafbb tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:46.559272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efab54b-16b8-44a6-b478-53080cd26f63 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a76c4316c64403b93159e5b87d994a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc0c828632bf46949b79badfb31e0b7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:46.560804 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efab54b-16b8-44a6-b478-53080cd26f63 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:46.561122 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efab54b-16b8-44a6-b478-53080cd26f63 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:46.561471 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efab54b-16b8-44a6-b478-53080cd26f63 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:46.562299 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efab54b-16b8-44a6-b478-53080cd26f63 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:34:46.562656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efab54b-16b8-44a6-b478-53080cd26f63 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:46.564598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efab54b-16b8-44a6-b478-53080cd26f63 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:46.597549 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 911/1829] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:34:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:34:46.609909 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7d131a1a-1491-488c-995b-be9fd76f25ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35e3ebe1ddc546bf8b10304e6417d4b5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:46.610665 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7d131a1a-1491-488c-995b-be9fd76f25ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35e3ebe1ddc546bf8b10304e6417d4b5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:46.621392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7d131a1a-1491-488c-995b-be9fd76f25ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35e3ebe1ddc546bf8b10304e6417d4b5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:46.681106 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efab54b-16b8-44a6-b478-53080cd26f63 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:46.708579 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efab54b-16b8-44a6-b478-53080cd26f63 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:46.723287 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6872988-5cf5-46f5-9ee1-ed8977fad3f4 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a76c4316c64403b93159e5b87d994a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fc0c828632bf46949b79badfb31e0b7e', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:46.724352 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6872988-5cf5-46f5-9ee1-ed8977fad3f4 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:46.724707 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6872988-5cf5-46f5-9ee1-ed8977fad3f4 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:46.725001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6872988-5cf5-46f5-9ee1-ed8977fad3f4 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] PATH_INFO: `/v3/projects/2de1fd3c1f1745dc903938b33a081787` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:46.726600 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6872988-5cf5-46f5-9ee1-ed8977fad3f4 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2de1fd3c1f1745dc903938b33a081787)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:46.728755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6872988-5cf5-46f5-9ee1-ed8977fad3f4 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:46.790849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7d131a1a-1491-488c-995b-be9fd76f25ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35e3ebe1ddc546bf8b10304e6417d4b5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:46.790849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-7d131a1a-1491-488c-995b-be9fd76f25ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35e3ebe1ddc546bf8b10304e6417d4b5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:46.833952 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 919/1830] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:34:46 2022] DELETE /identity/v3/users/35e3ebe1ddc546bf8b10304e6417d4b5 => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:46.965827 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d6872988-5cf5-46f5-9ee1-ed8977fad3f4 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2de1fd3c1f1745dc903938b33a081787', 'request_id': 'req-d6872988-5cf5-46f5-9ee1-ed8977fad3f4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:46.982724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d6872988-5cf5-46f5-9ee1-ed8977fad3f4 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2de1fd3c1f1745dc903938b33a081787', 'request_id': 'req-d6872988-5cf5-46f5-9ee1-ed8977fad3f4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:47.002952 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d6872988-5cf5-46f5-9ee1-ed8977fad3f4 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2de1fd3c1f1745dc903938b33a081787', 'request_id': 'req-d6872988-5cf5-46f5-9ee1-ed8977fad3f4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:47.164523 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d6872988-5cf5-46f5-9ee1-ed8977fad3f4 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] The token cache is being invalidate because project 2de1fd3c1f1745dc903938b33a081787 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:34:47.164523 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d6872988-5cf5-46f5-9ee1-ed8977fad3f4 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:47.165673 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d6872988-5cf5-46f5-9ee1-ed8977fad3f4 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:47.182778 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 912/1831] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:34:46 2022] DELETE /identity/v3/projects/2de1fd3c1f1745dc903938b33a081787 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:47.599305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:47.843536 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d131a1a-1491-488c-995b-be9fd76f25ce admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:47.847373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7d131a1a-1491-488c-995b-be9fd76f25ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:48.091149 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d131a1a-1491-488c-995b-be9fd76f25ce admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:48.221697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:48.309107 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab38ad-d11c-4bd0-8291-2394a4f38c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:48.310050 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab38ad-d11c-4bd0-8291-2394a4f38c2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:48.310776 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab38ad-d11c-4bd0-8291-2394a4f38c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:48.310776 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab38ad-d11c-4bd0-8291-2394a4f38c2e admin admin] PATH_INFO: `/v3/projects/d53f6811380a4832af1e2c436d97fbf3` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:48.312668 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab38ad-d11c-4bd0-8291-2394a4f38c2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d53f6811380a4832af1e2c436d97fbf3)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:48.315722 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab38ad-d11c-4bd0-8291-2394a4f38c2e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:48.653890 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-63ab38ad-d11c-4bd0-8291-2394a4f38c2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd53f6811380a4832af1e2c436d97fbf3', 'request_id': 'req-63ab38ad-d11c-4bd0-8291-2394a4f38c2e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:48.704174 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6872988-5cf5-46f5-9ee1-ed8977fad3f4 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:48.705866 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d6872988-5cf5-46f5-9ee1-ed8977fad3f4 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:48.731349 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6872988-5cf5-46f5-9ee1-ed8977fad3f4 tempest-VolumeQuotaClassesTest-1156602365 tempest-VolumeQuotaClassesTest-1156602365-project-admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:48.780500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-63ab38ad-d11c-4bd0-8291-2394a4f38c2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd53f6811380a4832af1e2c436d97fbf3', 'request_id': 'req-63ab38ad-d11c-4bd0-8291-2394a4f38c2e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:48.802976 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-63ab38ad-d11c-4bd0-8291-2394a4f38c2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd53f6811380a4832af1e2c436d97fbf3', 'request_id': 'req-63ab38ad-d11c-4bd0-8291-2394a4f38c2e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:48.851423 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b4c79-b355-4a09-a36e-2e5a3b1df92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:48.853569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b4c79-b355-4a09-a36e-2e5a3b1df92b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:48.855351 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b4c79-b355-4a09-a36e-2e5a3b1df92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:48.862070 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29b4c79-b355-4a09-a36e-2e5a3b1df92b admin admin] PATH_INFO: `/v3/users/b0775f2a2b924af28cad657ebf3e6f78` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:48.862070 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b4c79-b355-4a09-a36e-2e5a3b1df92b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0775f2a2b924af28cad657ebf3e6f78)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:48.862831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29b4c79-b355-4a09-a36e-2e5a3b1df92b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:48.878042 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-63ab38ad-d11c-4bd0-8291-2394a4f38c2e admin admin] The token cache is being invalidate because project d53f6811380a4832af1e2c436d97fbf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:34:48.878839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-63ab38ad-d11c-4bd0-8291-2394a4f38c2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:48.880952 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-63ab38ad-d11c-4bd0-8291-2394a4f38c2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:48.892845 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 920/1832] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:34:47 2022] DELETE /identity/v3/projects/d53f6811380a4832af1e2c436d97fbf3 => generated 0 bytes in 1051 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:48.901795 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2181a-9c1b-4526-8c16-90448e1e14a9 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:48.902093 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2181a-9c1b-4526-8c16-90448e1e14a9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:48.902380 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f2181a-9c1b-4526-8c16-90448e1e14a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:48.947113 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-99f2181a-9c1b-4526-8c16-90448e1e14a9 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:48.949395 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-99f2181a-9c1b-4526-8c16-90448e1e14a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:48.959777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 921/1833] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:48.980357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf14600-b361-4d4b-8c86-e38c8cbf77b1 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:48.980676 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf14600-b361-4d4b-8c86-e38c8cbf77b1 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:48.980913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf14600-b361-4d4b-8c86-e38c8cbf77b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:49.042447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-aaf14600-b361-4d4b-8c86-e38c8cbf77b1 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:49.043813 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-aaf14600-b361-4d4b-8c86-e38c8cbf77b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:49.055066 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 922/1834] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:49.058903 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf14600-b361-4d4b-8c86-e38c8cbf77b1 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:49.077254 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf14600-b361-4d4b-8c86-e38c8cbf77b1 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:49.094177 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703085f-3eef-4946-831b-6e4d85c26ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:49.094177 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703085f-3eef-4946-831b-6e4d85c26ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:49.094541 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703085f-3eef-4946-831b-6e4d85c26ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:49.094820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703085f-3eef-4946-831b-6e4d85c26ef2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:49.095896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d703085f-3eef-4946-831b-6e4d85c26ef2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:34:49.096100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d703085f-3eef-4946-831b-6e4d85c26ef2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:49.097869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d703085f-3eef-4946-831b-6e4d85c26ef2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:49.115308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 923/1835] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:34:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:34:49.116901 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703085f-3eef-4946-831b-6e4d85c26ef2 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:49.119428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f29b4c79-b355-4a09-a36e-2e5a3b1df92b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0775f2a2b924af28cad657ebf3e6f78'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:49.122470 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d703085f-3eef-4946-831b-6e4d85c26ef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:49.122626 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f29b4c79-b355-4a09-a36e-2e5a3b1df92b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0775f2a2b924af28cad657ebf3e6f78'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:49.138790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703085f-3eef-4946-831b-6e4d85c26ef2 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:49.145812 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee575d-5131-41e4-aaeb-0c8139bb4a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:49.147087 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee575d-5131-41e4-aaeb-0c8139bb4a1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:49.147795 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee575d-5131-41e4-aaeb-0c8139bb4a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:49.148489 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee575d-5131-41e4-aaeb-0c8139bb4a1b admin admin] PATH_INFO: `/v3/users/dc4c24a998354c18a889c3a9f5f62a37` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:49.150070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee575d-5131-41e4-aaeb-0c8139bb4a1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc4c24a998354c18a889c3a9f5f62a37)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:49.152764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dee575d-5131-41e4-aaeb-0c8139bb4a1b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:49.211829 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f29b4c79-b355-4a09-a36e-2e5a3b1df92b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0775f2a2b924af28cad657ebf3e6f78'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:49.231368 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f29b4c79-b355-4a09-a36e-2e5a3b1df92b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0775f2a2b924af28cad657ebf3e6f78'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:49.232763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-f29b4c79-b355-4a09-a36e-2e5a3b1df92b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0775f2a2b924af28cad657ebf3e6f78'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:49.275972 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 913/1836] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:34:48 2022] DELETE /identity/v3/users/b0775f2a2b924af28cad657ebf3e6f78 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:49.277998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b4c79-b355-4a09-a36e-2e5a3b1df92b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:49.283989 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f29b4c79-b355-4a09-a36e-2e5a3b1df92b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:49.418519 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0dee575d-5131-41e4-aaeb-0c8139bb4a1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc4c24a998354c18a889c3a9f5f62a37'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:49.419449 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0dee575d-5131-41e4-aaeb-0c8139bb4a1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc4c24a998354c18a889c3a9f5f62a37'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:49.424909 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29b4c79-b355-4a09-a36e-2e5a3b1df92b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:49.430244 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7facc-d1c9-40f4-9c08-36a01f74e538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:49.431035 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7facc-d1c9-40f4-9c08-36a01f74e538 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:49.431643 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7facc-d1c9-40f4-9c08-36a01f74e538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:49.431811 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7facc-d1c9-40f4-9c08-36a01f74e538 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:49.432672 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7facc-d1c9-40f4-9c08-36a01f74e538 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:49.434721 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7facc-d1c9-40f4-9c08-36a01f74e538 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:49.439598 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0dee575d-5131-41e4-aaeb-0c8139bb4a1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc4c24a998354c18a889c3a9f5f62a37'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:49.506898 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0dee575d-5131-41e4-aaeb-0c8139bb4a1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc4c24a998354c18a889c3a9f5f62a37'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:49.507813 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0dee575d-5131-41e4-aaeb-0c8139bb4a1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc4c24a998354c18a889c3a9f5f62a37'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:49.539677 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 924/1837] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:34:49 2022] DELETE /identity/v3/users/dc4c24a998354c18a889c3a9f5f62a37 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:49.539677 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bbad8-c29f-4bc8-82ca-aec0668cde87 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:49.539677 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bbad8-c29f-4bc8-82ca-aec0668cde87 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:49.539677 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bbad8-c29f-4bc8-82ca-aec0668cde87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:49.565616 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-352bbad8-c29f-4bc8-82ca-aec0668cde87 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:49.567202 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-352bbad8-c29f-4bc8-82ca-aec0668cde87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:49.586792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 914/1838] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:34:49 2022] POST /identity/v3/projects => generated 385 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:49.589980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998c89a-ad2f-45b8-b592-a5f33ae83845 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:49.590836 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998c89a-ad2f-45b8-b592-a5f33ae83845 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:49.591416 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998c89a-ad2f-45b8-b592-a5f33ae83845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:49.623870 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b998c89a-ad2f-45b8-b592-a5f33ae83845 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:49.627871 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b998c89a-ad2f-45b8-b592-a5f33ae83845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:49.649990 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 925/1839] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:49.657054 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352bbad8-c29f-4bc8-82ca-aec0668cde87 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:49.660722 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-352bbad8-c29f-4bc8-82ca-aec0668cde87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:49.695344 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352bbad8-c29f-4bc8-82ca-aec0668cde87 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:49.743788 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 915/1840] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:49.745530 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998c89a-ad2f-45b8-b592-a5f33ae83845 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:49.747284 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b998c89a-ad2f-45b8-b592-a5f33ae83845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:49.793423 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045babe-9f0c-4273-ace5-4de6c84902ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:49.794164 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045babe-9f0c-4273-ace5-4de6c84902ab admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:49.794449 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045babe-9f0c-4273-ace5-4de6c84902ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:49.794938 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045babe-9f0c-4273-ace5-4de6c84902ab admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:49.796069 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c045babe-9f0c-4273-ace5-4de6c84902ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:49.798848 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c045babe-9f0c-4273-ace5-4de6c84902ab admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:49.846866 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998c89a-ad2f-45b8-b592-a5f33ae83845 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:49.853418 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a0e18-7502-4d62-9a83-2a7ab08d0279 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:49.854617 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a0e18-7502-4d62-9a83-2a7ab08d0279 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:49.855083 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a0e18-7502-4d62-9a83-2a7ab08d0279 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:49.855625 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a0e18-7502-4d62-9a83-2a7ab08d0279 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:49.874486 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5a0e18-7502-4d62-9a83-2a7ab08d0279 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:49.877484 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5a0e18-7502-4d62-9a83-2a7ab08d0279 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:49.896059 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 916/1841] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:34:49.897569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a0e18-7502-4d62-9a83-2a7ab08d0279 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:49.915078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a0e18-7502-4d62-9a83-2a7ab08d0279 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:49.925889 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 926/1842] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:34:49 2022] POST /identity/v3/users => generated 293 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:49.926633 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c629a33-d891-4e7e-80e3-6560bb741073 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:49.926633 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c629a33-d891-4e7e-80e3-6560bb741073 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:49.926633 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c629a33-d891-4e7e-80e3-6560bb741073 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:49.927018 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c629a33-d891-4e7e-80e3-6560bb741073 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:49.929254 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045babe-9f0c-4273-ace5-4de6c84902ab admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:49.942714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045babe-9f0c-4273-ace5-4de6c84902ab admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:49.951907 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c629a33-d891-4e7e-80e3-6560bb741073 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:49.954027 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921cd45-8a43-4067-9809-59b44c9c481b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:49.954807 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c629a33-d891-4e7e-80e3-6560bb741073 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:49.955263 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921cd45-8a43-4067-9809-59b44c9c481b admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:49.956679 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921cd45-8a43-4067-9809-59b44c9c481b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:49.956983 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921cd45-8a43-4067-9809-59b44c9c481b admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:49.957761 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e921cd45-8a43-4067-9809-59b44c9c481b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:49.961631 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e921cd45-8a43-4067-9809-59b44c9c481b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:49.974113 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 917/1843] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:34:49.985954 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 927/1844] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:49.992263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c629a33-d891-4e7e-80e3-6560bb741073 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:50.007678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c629a33-d891-4e7e-80e3-6560bb741073 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:50.018508 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57988a03-09af-472a-9025-75968aaccdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:50.019654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57988a03-09af-472a-9025-75968aaccdfb admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:50.020310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57988a03-09af-472a-9025-75968aaccdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:50.021546 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57988a03-09af-472a-9025-75968aaccdfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:50.022054 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57988a03-09af-472a-9025-75968aaccdfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:50.024287 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57988a03-09af-472a-9025-75968aaccdfb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:50.039017 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 918/1845] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:50.045569 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921cd45-8a43-4067-9809-59b44c9c481b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:50.059204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921cd45-8a43-4067-9809-59b44c9c481b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:50.066084 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7096ef-8545-4669-8b9b-56e877193549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:50.067021 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7096ef-8545-4669-8b9b-56e877193549 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:50.067311 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7096ef-8545-4669-8b9b-56e877193549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:50.067658 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7096ef-8545-4669-8b9b-56e877193549 admin admin] PATH_INFO: `/v3/projects/2ec97a5a23174830a41a272d53d6de5c/users/24a7c6eef5284d1e86531b6a5dfe63e2/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:50.092563 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7096ef-8545-4669-8b9b-56e877193549 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ec97a5a23174830a41a272d53d6de5c, user_id=24a7c6eef5284d1e86531b6a5dfe63e2, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:50.095968 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7096ef-8545-4669-8b9b-56e877193549 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:50.275898 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 928/1846] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:50 2022] PUT /identity/v3/projects/2ec97a5a23174830a41a272d53d6de5c/users/24a7c6eef5284d1e86531b6a5dfe63e2/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:50.321499 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f20107-6956-4434-8b04-3712b28ed55e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:50.321948 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f20107-6956-4434-8b04-3712b28ed55e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:50.322389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f20107-6956-4434-8b04-3712b28ed55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:50.356768 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-13f20107-6956-4434-8b04-3712b28ed55e None None] MFA Rules not processed for user `24a7c6eef5284d1e86531b6a5dfe63e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:50.358187 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-13f20107-6956-4434-8b04-3712b28ed55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:50.454514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 919/1847] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:34:50.459463 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7096ef-8545-4669-8b9b-56e877193549 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:50.580367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7096ef-8545-4669-8b9b-56e877193549 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:50.603236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33812e37-b9ec-417e-9e1a-e15d9a6f7b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:50.604086 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33812e37-b9ec-417e-9e1a-e15d9a6f7b92 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:50.604790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33812e37-b9ec-417e-9e1a-e15d9a6f7b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:50.604790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33812e37-b9ec-417e-9e1a-e15d9a6f7b92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:50.605519 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33812e37-b9ec-417e-9e1a-e15d9a6f7b92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:50.608600 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33812e37-b9ec-417e-9e1a-e15d9a6f7b92 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:50.728512 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 929/1848] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:34:50 2022] POST /identity/v3/projects => generated 385 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:50.729790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:50.733117 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f20107-6956-4434-8b04-3712b28ed55e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:50.767373 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f20107-6956-4434-8b04-3712b28ed55e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:50.829402 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33812e37-b9ec-417e-9e1a-e15d9a6f7b92 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:50.834262 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-33812e37-b9ec-417e-9e1a-e15d9a6f7b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:50.858501 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:50.912629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc297d63-6380-483a-8055-a112c39e278a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:50.913733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc297d63-6380-483a-8055-a112c39e278a admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:50.914063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc297d63-6380-483a-8055-a112c39e278a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:50.914344 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc297d63-6380-483a-8055-a112c39e278a admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:50.916880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc297d63-6380-483a-8055-a112c39e278a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:50.920387 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc297d63-6380-483a-8055-a112c39e278a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:50.964763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33812e37-b9ec-417e-9e1a-e15d9a6f7b92 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:50.973153 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66fee79-47f2-4714-a57c-e26f2b800ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:50.974044 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fee79-47f2-4714-a57c-e26f2b800ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:50.974325 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fee79-47f2-4714-a57c-e26f2b800ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:50.974929 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66fee79-47f2-4714-a57c-e26f2b800ef7 admin admin] PATH_INFO: `/v3/projects/d353005498fc443dbb6026b2599347ea` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:50.976273 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66fee79-47f2-4714-a57c-e26f2b800ef7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d353005498fc443dbb6026b2599347ea)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:50.978365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66fee79-47f2-4714-a57c-e26f2b800ef7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:51.057486 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 920/1849] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:34:50 2022] POST /identity/v3/users => generated 299 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:51.058705 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc297d63-6380-483a-8055-a112c39e278a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:51.060369 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fc297d63-6380-483a-8055-a112c39e278a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:51.072453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc297d63-6380-483a-8055-a112c39e278a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:51.077440 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4300b81-71fb-4d06-a5fc-74d4ea7fc871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:51.079221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4300b81-71fb-4d06-a5fc-74d4ea7fc871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:51.079597 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4300b81-71fb-4d06-a5fc-74d4ea7fc871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:51.079883 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4300b81-71fb-4d06-a5fc-74d4ea7fc871 admin admin] PATH_INFO: `/v3/projects/cd8720b566d44431bd0848c7d6f4ca5c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:51.081348 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4300b81-71fb-4d06-a5fc-74d4ea7fc871 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd8720b566d44431bd0848c7d6f4ca5c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:51.083237 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4300b81-71fb-4d06-a5fc-74d4ea7fc871 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:51.135204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d66fee79-47f2-4714-a57c-e26f2b800ef7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd353005498fc443dbb6026b2599347ea', 'request_id': 'req-d66fee79-47f2-4714-a57c-e26f2b800ef7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:51.245775 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d66fee79-47f2-4714-a57c-e26f2b800ef7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd353005498fc443dbb6026b2599347ea', 'request_id': 'req-d66fee79-47f2-4714-a57c-e26f2b800ef7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:51.253906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d66fee79-47f2-4714-a57c-e26f2b800ef7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd353005498fc443dbb6026b2599347ea', 'request_id': 'req-d66fee79-47f2-4714-a57c-e26f2b800ef7'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:51.258846 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e4300b81-71fb-4d06-a5fc-74d4ea7fc871 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd8720b566d44431bd0848c7d6f4ca5c', 'request_id': 'req-e4300b81-71fb-4d06-a5fc-74d4ea7fc871'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:51.266440 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d66fee79-47f2-4714-a57c-e26f2b800ef7 admin admin] The token cache is being invalidate because project d353005498fc443dbb6026b2599347ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:34:51.267142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d66fee79-47f2-4714-a57c-e26f2b800ef7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:51.268955 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d66fee79-47f2-4714-a57c-e26f2b800ef7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:51.274666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 930/1850] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:34:50 2022] DELETE /identity/v3/projects/d353005498fc443dbb6026b2599347ea => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:51.274796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e4300b81-71fb-4d06-a5fc-74d4ea7fc871 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd8720b566d44431bd0848c7d6f4ca5c', 'request_id': 'req-e4300b81-71fb-4d06-a5fc-74d4ea7fc871'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:51.274893 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66fee79-47f2-4714-a57c-e26f2b800ef7 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:51.274893 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d66fee79-47f2-4714-a57c-e26f2b800ef7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:51.283121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e4300b81-71fb-4d06-a5fc-74d4ea7fc871 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd8720b566d44431bd0848c7d6f4ca5c', 'request_id': 'req-e4300b81-71fb-4d06-a5fc-74d4ea7fc871'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:51.386692 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66fee79-47f2-4714-a57c-e26f2b800ef7 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:51.620170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479337e-9d9a-4875-8772-7ef08338d3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:51.620170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479337e-9d9a-4875-8772-7ef08338d3f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:51.620170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479337e-9d9a-4875-8772-7ef08338d3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:51.620170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479337e-9d9a-4875-8772-7ef08338d3f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:51.621504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479337e-9d9a-4875-8772-7ef08338d3f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:51.621504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479337e-9d9a-4875-8772-7ef08338d3f8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:51.621504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 931/1851] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:51.621504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479337e-9d9a-4875-8772-7ef08338d3f8 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:51.621504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5479337e-9d9a-4875-8772-7ef08338d3f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:51.621504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479337e-9d9a-4875-8772-7ef08338d3f8 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:51.622875 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6129ce-12ed-4068-beb0-7f301b44565a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:51.622875 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6129ce-12ed-4068-beb0-7f301b44565a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:51.622875 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6129ce-12ed-4068-beb0-7f301b44565a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:51.622875 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6129ce-12ed-4068-beb0-7f301b44565a admin admin] PATH_INFO: `/v3/users/8a76c4316c64403b93159e5b87d994a0` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:51.623615 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6129ce-12ed-4068-beb0-7f301b44565a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a76c4316c64403b93159e5b87d994a0)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:51.623615 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6129ce-12ed-4068-beb0-7f301b44565a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:51.647070 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e4300b81-71fb-4d06-a5fc-74d4ea7fc871 admin admin] The token cache is being invalidate because project cd8720b566d44431bd0848c7d6f4ca5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:34:51.653921 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e4300b81-71fb-4d06-a5fc-74d4ea7fc871 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:51.653921 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e4300b81-71fb-4d06-a5fc-74d4ea7fc871 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:51.653921 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 921/1852] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:34:51 2022] DELETE /identity/v3/projects/cd8720b566d44431bd0848c7d6f4ca5c => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:51.653921 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4300b81-71fb-4d06-a5fc-74d4ea7fc871 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:51.653921 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e4300b81-71fb-4d06-a5fc-74d4ea7fc871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:51.670749 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4300b81-71fb-4d06-a5fc-74d4ea7fc871 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:51.678712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e414a2-9de5-4ab5-8bc2-5d6f9a0a7efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:51.679672 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e414a2-9de5-4ab5-8bc2-5d6f9a0a7efc admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:51.679970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e414a2-9de5-4ab5-8bc2-5d6f9a0a7efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:51.680230 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e414a2-9de5-4ab5-8bc2-5d6f9a0a7efc admin admin] PATH_INFO: `/v3/projects/1caa5b0935204192b4520d37909f5fcb/users/c5f4ac61ec774888a9c1a4ad963d0359/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:51.698522 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e414a2-9de5-4ab5-8bc2-5d6f9a0a7efc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1caa5b0935204192b4520d37909f5fcb, user_id=c5f4ac61ec774888a9c1a4ad963d0359, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:51.711779 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e414a2-9de5-4ab5-8bc2-5d6f9a0a7efc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:51.876597 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 922/1853] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:51 2022] PUT /identity/v3/projects/1caa5b0935204192b4520d37909f5fcb/users/c5f4ac61ec774888a9c1a4ad963d0359/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:51.877908 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e414a2-9de5-4ab5-8bc2-5d6f9a0a7efc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:51.888785 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-17e414a2-9de5-4ab5-8bc2-5d6f9a0a7efc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:52.029598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e414a2-9de5-4ab5-8bc2-5d6f9a0a7efc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:52.039014 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a557e8c7-4fd9-4033-91cc-a669158136b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:52.041991 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2d6129ce-12ed-4068-beb0-7f301b44565a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a76c4316c64403b93159e5b87d994a0'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:52.043280 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2d6129ce-12ed-4068-beb0-7f301b44565a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a76c4316c64403b93159e5b87d994a0'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:52.049116 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557e8c7-4fd9-4033-91cc-a669158136b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:52.049511 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557e8c7-4fd9-4033-91cc-a669158136b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:52.049802 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557e8c7-4fd9-4033-91cc-a669158136b4 admin admin] PATH_INFO: `/v3/users/8a35c6456fdb4b4ba650194bdedf2b52` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:52.052210 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a557e8c7-4fd9-4033-91cc-a669158136b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a35c6456fdb4b4ba650194bdedf2b52)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:52.054342 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a557e8c7-4fd9-4033-91cc-a669158136b4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:52.057735 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2d6129ce-12ed-4068-beb0-7f301b44565a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a76c4316c64403b93159e5b87d994a0'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:52.189265 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2d6129ce-12ed-4068-beb0-7f301b44565a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a76c4316c64403b93159e5b87d994a0'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:52.189922 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-2d6129ce-12ed-4068-beb0-7f301b44565a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a76c4316c64403b93159e5b87d994a0'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:52.220461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 932/1854] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:34:51 2022] DELETE /identity/v3/users/8a76c4316c64403b93159e5b87d994a0 => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:52.221532 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6129ce-12ed-4068-beb0-7f301b44565a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:52.222993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2d6129ce-12ed-4068-beb0-7f301b44565a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:52.309213 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6129ce-12ed-4068-beb0-7f301b44565a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:52.314763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea348f6e-ad54-4a3e-ba5a-894bc93409c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:52.315401 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea348f6e-ad54-4a3e-ba5a-894bc93409c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:52.315759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea348f6e-ad54-4a3e-ba5a-894bc93409c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:52.316107 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea348f6e-ad54-4a3e-ba5a-894bc93409c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:52.316789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea348f6e-ad54-4a3e-ba5a-894bc93409c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:52.318454 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea348f6e-ad54-4a3e-ba5a-894bc93409c7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:52.321140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a557e8c7-4fd9-4033-91cc-a669158136b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a35c6456fdb4b4ba650194bdedf2b52'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:52.321967 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a557e8c7-4fd9-4033-91cc-a669158136b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a35c6456fdb4b4ba650194bdedf2b52'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:52.335497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 933/1855] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:52.339136 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea348f6e-ad54-4a3e-ba5a-894bc93409c7 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:52.352253 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea348f6e-ad54-4a3e-ba5a-894bc93409c7 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:52.357933 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66662500-ce07-4288-9038-0246be4028d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:52.359161 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66662500-ce07-4288-9038-0246be4028d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:52.360365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66662500-ce07-4288-9038-0246be4028d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:52.360829 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66662500-ce07-4288-9038-0246be4028d1 admin admin] PATH_INFO: `/v3/projects/1caa5b0935204192b4520d37909f5fcb/users/c5f4ac61ec774888a9c1a4ad963d0359/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:52.363166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66662500-ce07-4288-9038-0246be4028d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1caa5b0935204192b4520d37909f5fcb, user_id=c5f4ac61ec774888a9c1a4ad963d0359, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:52.366296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66662500-ce07-4288-9038-0246be4028d1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:52.379408 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 934/1856] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:52 2022] PUT /identity/v3/projects/1caa5b0935204192b4520d37909f5fcb/users/c5f4ac61ec774888a9c1a4ad963d0359/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:52.404369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208e859-021c-4ab0-8fb1-9d199f90f0ee None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:52.404369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208e859-021c-4ab0-8fb1-9d199f90f0ee None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:52.404369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208e859-021c-4ab0-8fb1-9d199f90f0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:52.431197 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e208e859-021c-4ab0-8fb1-9d199f90f0ee None None] MFA Rules not processed for user `c5f4ac61ec774888a9c1a4ad963d0359`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:52.432520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e208e859-021c-4ab0-8fb1-9d199f90f0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:52.436001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a557e8c7-4fd9-4033-91cc-a669158136b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a35c6456fdb4b4ba650194bdedf2b52'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:52.446065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a557e8c7-4fd9-4033-91cc-a669158136b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a35c6456fdb4b4ba650194bdedf2b52'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:52.447659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a557e8c7-4fd9-4033-91cc-a669158136b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a35c6456fdb4b4ba650194bdedf2b52'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:52.474850 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 923/1857] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:34:51 2022] DELETE /identity/v3/users/8a35c6456fdb4b4ba650194bdedf2b52 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:52.540918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 935/1858] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:52 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:34:52.555300 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a557e8c7-4fd9-4033-91cc-a669158136b4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:52.557762 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a557e8c7-4fd9-4033-91cc-a669158136b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:52.638213 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a557e8c7-4fd9-4033-91cc-a669158136b4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:52.643923 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3579482-9753-47c1-b78c-740faebb1ae6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:52.644604 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3579482-9753-47c1-b78c-740faebb1ae6 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:52.645149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3579482-9753-47c1-b78c-740faebb1ae6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:52.645268 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3579482-9753-47c1-b78c-740faebb1ae6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:52.647655 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a3579482-9753-47c1-b78c-740faebb1ae6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:52.729046 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3579482-9753-47c1-b78c-740faebb1ae6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:52.730441 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3579482-9753-47c1-b78c-740faebb1ae6 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:52.743862 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 924/1859] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:34:52.895517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208e859-021c-4ab0-8fb1-9d199f90f0ee None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:52.899380 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e208e859-021c-4ab0-8fb1-9d199f90f0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:53.045208 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:53.094169 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208e859-021c-4ab0-8fb1-9d199f90f0ee None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:53.176740 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f382db51-7d2d-45c7-8ee8-8ca30c169ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:53.176740 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f382db51-7d2d-45c7-8ee8-8ca30c169ace admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:53.176740 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f382db51-7d2d-45c7-8ee8-8ca30c169ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:53.176740 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f382db51-7d2d-45c7-8ee8-8ca30c169ace admin admin] PATH_INFO: `/v3/projects/fc0c828632bf46949b79badfb31e0b7e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:53.177994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f382db51-7d2d-45c7-8ee8-8ca30c169ace admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc0c828632bf46949b79badfb31e0b7e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:53.180416 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f382db51-7d2d-45c7-8ee8-8ca30c169ace admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:53.320442 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3579482-9753-47c1-b78c-740faebb1ae6 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:53.321972 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a3579482-9753-47c1-b78c-740faebb1ae6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:53.340569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3579482-9753-47c1-b78c-740faebb1ae6 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:53.360650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1f1e5-b7d7-4f22-96da-3c972863704e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:53.360650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1f1e5-b7d7-4f22-96da-3c972863704e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:53.360650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1f1e5-b7d7-4f22-96da-3c972863704e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:53.360650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1f1e5-b7d7-4f22-96da-3c972863704e admin admin] PATH_INFO: `/v3/projects/36b7f62cfa2c42cf94244f6d4142d397` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:53.361737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1f1e5-b7d7-4f22-96da-3c972863704e admin admin] RBAC: Authorizing `identity:delete_project(project_id=36b7f62cfa2c42cf94244f6d4142d397)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:53.361737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1f1e5-b7d7-4f22-96da-3c972863704e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:53.398746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f382db51-7d2d-45c7-8ee8-8ca30c169ace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc0c828632bf46949b79badfb31e0b7e', 'request_id': 'req-f382db51-7d2d-45c7-8ee8-8ca30c169ace'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:53.479784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f382db51-7d2d-45c7-8ee8-8ca30c169ace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc0c828632bf46949b79badfb31e0b7e', 'request_id': 'req-f382db51-7d2d-45c7-8ee8-8ca30c169ace'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:53.481275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:53.489831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f382db51-7d2d-45c7-8ee8-8ca30c169ace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc0c828632bf46949b79badfb31e0b7e', 'request_id': 'req-f382db51-7d2d-45c7-8ee8-8ca30c169ace'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:53.503235 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f382db51-7d2d-45c7-8ee8-8ca30c169ace admin admin] The token cache is being invalidate because project fc0c828632bf46949b79badfb31e0b7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:34:53.506673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f382db51-7d2d-45c7-8ee8-8ca30c169ace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:53.506673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f382db51-7d2d-45c7-8ee8-8ca30c169ace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:53.506673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 936/1860] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:34:52 2022] DELETE /identity/v3/projects/fc0c828632bf46949b79badfb31e0b7e => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:53.511207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09169f02-df21-4a18-a6e5-276c9af509c6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:53.511459 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09169f02-df21-4a18-a6e5-276c9af509c6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:53.511838 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09169f02-df21-4a18-a6e5-276c9af509c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:53.513180 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b6a1f1e5-b7d7-4f22-96da-3c972863704e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36b7f62cfa2c42cf94244f6d4142d397', 'request_id': 'req-b6a1f1e5-b7d7-4f22-96da-3c972863704e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:53.525006 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b6a1f1e5-b7d7-4f22-96da-3c972863704e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36b7f62cfa2c42cf94244f6d4142d397', 'request_id': 'req-b6a1f1e5-b7d7-4f22-96da-3c972863704e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:53.534412 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-09169f02-df21-4a18-a6e5-276c9af509c6 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:53.535260 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b6a1f1e5-b7d7-4f22-96da-3c972863704e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36b7f62cfa2c42cf94244f6d4142d397', 'request_id': 'req-b6a1f1e5-b7d7-4f22-96da-3c972863704e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:53.535703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-09169f02-df21-4a18-a6e5-276c9af509c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:53.618271 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 937/1861] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:53.618591 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b6a1f1e5-b7d7-4f22-96da-3c972863704e admin admin] The token cache is being invalidate because project 36b7f62cfa2c42cf94244f6d4142d397 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:34:53.618591 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b6a1f1e5-b7d7-4f22-96da-3c972863704e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:53.619040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b6a1f1e5-b7d7-4f22-96da-3c972863704e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:34:53.621583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 925/1862] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:34:53 2022] DELETE /identity/v3/projects/36b7f62cfa2c42cf94244f6d4142d397 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:34:53.622659 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d127ab-f62e-4cbf-9cd4-a9d976c402e6 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:53.623127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d127ab-f62e-4cbf-9cd4-a9d976c402e6 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:53.623641 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d127ab-f62e-4cbf-9cd4-a9d976c402e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:53.627845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c57fb-83d1-431c-b446-c724fd52e781 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:53.628586 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c57fb-83d1-431c-b446-c724fd52e781 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:53.629123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c57fb-83d1-431c-b446-c724fd52e781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:53.654921 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-96d127ab-f62e-4cbf-9cd4-a9d976c402e6 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:53.657953 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-96d127ab-f62e-4cbf-9cd4-a9d976c402e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:53.659882 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3e4c57fb-83d1-431c-b446-c724fd52e781 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:53.662208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3e4c57fb-83d1-431c-b446-c724fd52e781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:53.671337 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 938/1863] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:53.678714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d127ab-f62e-4cbf-9cd4-a9d976c402e6 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:53.681471 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 926/1864] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:53.691304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12643e-d62a-4bdc-877d-f6ada67a9c40 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:53.691770 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12643e-d62a-4bdc-877d-f6ada67a9c40 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:53.692020 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d127ab-f62e-4cbf-9cd4-a9d976c402e6 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:53.692369 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12643e-d62a-4bdc-877d-f6ada67a9c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:53.698343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72bc384-695b-49b6-a09f-8f660ec29889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:53.699033 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bc384-695b-49b6-a09f-8f660ec29889 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:53.699316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bc384-695b-49b6-a09f-8f660ec29889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:53.699626 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bc384-695b-49b6-a09f-8f660ec29889 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:53.701635 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72bc384-695b-49b6-a09f-8f660ec29889 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:34:53.701809 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72bc384-695b-49b6-a09f-8f660ec29889 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:53.703411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72bc384-695b-49b6-a09f-8f660ec29889 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:53.716044 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4a12643e-d62a-4bdc-877d-f6ada67a9c40 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:53.719804 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4a12643e-d62a-4bdc-877d-f6ada67a9c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:53.735318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 927/1865] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:53.736961 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12643e-d62a-4bdc-877d-f6ada67a9c40 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:53.755849 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12643e-d62a-4bdc-877d-f6ada67a9c40 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:53.758808 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 939/1866] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:34:53.768077 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72bc384-695b-49b6-a09f-8f660ec29889 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:53.775195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110885a-0a3f-4be6-bf2a-b2bf1ff6da0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:53.776843 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110885a-0a3f-4be6-bf2a-b2bf1ff6da0e admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:53.777992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110885a-0a3f-4be6-bf2a-b2bf1ff6da0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:53.785002 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a110885a-0a3f-4be6-bf2a-b2bf1ff6da0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:53.786866 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a110885a-0a3f-4be6-bf2a-b2bf1ff6da0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:34:53.787543 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a110885a-0a3f-4be6-bf2a-b2bf1ff6da0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:53.794913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a110885a-0a3f-4be6-bf2a-b2bf1ff6da0e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:53.808758 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72bc384-695b-49b6-a09f-8f660ec29889 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:53.819529 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef6ffb-29ba-426a-8dc0-a585812ccaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:53.820444 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef6ffb-29ba-426a-8dc0-a585812ccaf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:53.825699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef6ffb-29ba-426a-8dc0-a585812ccaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:53.825699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef6ffb-29ba-426a-8dc0-a585812ccaf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:53.825699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ef6ffb-29ba-426a-8dc0-a585812ccaf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:53.825699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ef6ffb-29ba-426a-8dc0-a585812ccaf7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:53.838673 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 928/1867] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:34:53.846332 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110885a-0a3f-4be6-bf2a-b2bf1ff6da0e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:53.860381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a110885a-0a3f-4be6-bf2a-b2bf1ff6da0e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:53.866265 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad717e-aeec-42d1-851e-7d74e8e010a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:53.867153 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad717e-aeec-42d1-851e-7d74e8e010a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:53.867700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad717e-aeec-42d1-851e-7d74e8e010a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:53.867815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad717e-aeec-42d1-851e-7d74e8e010a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:53.871407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad717e-aeec-42d1-851e-7d74e8e010a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:53.871407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad717e-aeec-42d1-851e-7d74e8e010a5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:53.958538 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 940/1868] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:34:53 2022] POST /identity/v3/projects => generated 383 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:53.959960 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef6ffb-29ba-426a-8dc0-a585812ccaf7 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:53.993317 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 929/1869] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:34:53 2022] POST /identity/v3/projects => generated 355 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:53.994210 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad717e-aeec-42d1-851e-7d74e8e010a5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:54.076472 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef6ffb-29ba-426a-8dc0-a585812ccaf7 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:54.113837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad717e-aeec-42d1-851e-7d74e8e010a5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:54.120462 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eccca2e-7ff6-4639-af9d-1572b69e15ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:54.121271 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccca2e-7ff6-4639-af9d-1572b69e15ca admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:54.122848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccca2e-7ff6-4639-af9d-1572b69e15ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:54.123400 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccca2e-7ff6-4639-af9d-1572b69e15ca admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:54.124184 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eccca2e-7ff6-4639-af9d-1572b69e15ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:54.127215 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eccca2e-7ff6-4639-af9d-1572b69e15ca admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:54.148864 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:54.211759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad3d4c-87ce-4b7f-8d8d-9b0d9b7eb534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:54.215428 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad3d4c-87ce-4b7f-8d8d-9b0d9b7eb534 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:54.215428 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad3d4c-87ce-4b7f-8d8d-9b0d9b7eb534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:54.215428 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad3d4c-87ce-4b7f-8d8d-9b0d9b7eb534 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:54.221084 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad3d4c-87ce-4b7f-8d8d-9b0d9b7eb534 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:54.223585 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad3d4c-87ce-4b7f-8d8d-9b0d9b7eb534 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:54.274723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 930/1870] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:34:53 2022] POST /identity/v3/users => generated 278 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:54.281842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eccca2e-7ff6-4639-af9d-1572b69e15ca admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:54.296658 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eccca2e-7ff6-4639-af9d-1572b69e15ca admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:54.302494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44478fd-cb27-47c4-932e-8af63b6a50c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:54.302809 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44478fd-cb27-47c4-932e-8af63b6a50c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:54.303045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44478fd-cb27-47c4-932e-8af63b6a50c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:54.303275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44478fd-cb27-47c4-932e-8af63b6a50c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:54.309419 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44478fd-cb27-47c4-932e-8af63b6a50c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:54.309419 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44478fd-cb27-47c4-932e-8af63b6a50c9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:54.330280 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 931/1871] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:54.331537 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44478fd-cb27-47c4-932e-8af63b6a50c9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:54.343077 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44478fd-cb27-47c4-932e-8af63b6a50c9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:54.348472 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf4ed9-031e-4c53-aeb3-0458fd17cd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:54.349162 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf4ed9-031e-4c53-aeb3-0458fd17cd6c admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:54.349456 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf4ed9-031e-4c53-aeb3-0458fd17cd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:54.349725 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf4ed9-031e-4c53-aeb3-0458fd17cd6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:54.350646 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf4ed9-031e-4c53-aeb3-0458fd17cd6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:54.352379 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf4ed9-031e-4c53-aeb3-0458fd17cd6c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:54.366756 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 941/1872] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:34:53 2022] POST /identity/v3/users => generated 292 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:54.367024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 932/1873] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:54.369033 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad3d4c-87ce-4b7f-8d8d-9b0d9b7eb534 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:54.370301 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf4ed9-031e-4c53-aeb3-0458fd17cd6c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:54.382296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf4ed9-031e-4c53-aeb3-0458fd17cd6c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:54.395271 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1493c0-9afd-4819-be85-fef5e71ad94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:54.397553 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1493c0-9afd-4819-be85-fef5e71ad94e admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:54.398038 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1493c0-9afd-4819-be85-fef5e71ad94e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:54.398567 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1493c0-9afd-4819-be85-fef5e71ad94e admin admin] PATH_INFO: `/v3/projects/a6ea5cc4903842bc9c97a413a73a115b/users/bdfb7e5a13be4fdbb124b308647b823f/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:54.405628 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad3d4c-87ce-4b7f-8d8d-9b0d9b7eb534 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:54.428115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f943086-a6eb-4c48-83d0-705d3cd5f5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:54.429461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f943086-a6eb-4c48-83d0-705d3cd5f5ee admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:54.429743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f943086-a6eb-4c48-83d0-705d3cd5f5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:54.430446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f943086-a6eb-4c48-83d0-705d3cd5f5ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:54.430816 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1493c0-9afd-4819-be85-fef5e71ad94e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6ea5cc4903842bc9c97a413a73a115b, user_id=bdfb7e5a13be4fdbb124b308647b823f, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:54.431552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f943086-a6eb-4c48-83d0-705d3cd5f5ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:54.435074 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1493c0-9afd-4819-be85-fef5e71ad94e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:54.436626 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f943086-a6eb-4c48-83d0-705d3cd5f5ee admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:54.460796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 942/1874] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:54.468837 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f943086-a6eb-4c48-83d0-705d3cd5f5ee admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:54.482844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f943086-a6eb-4c48-83d0-705d3cd5f5ee admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:54.490693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9d6e0-c6ab-421c-8405-faa2bad547af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:54.491707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9d6e0-c6ab-421c-8405-faa2bad547af admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:54.492063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9d6e0-c6ab-421c-8405-faa2bad547af admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:54.492356 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9d6e0-c6ab-421c-8405-faa2bad547af admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:54.493671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c9d6e0-c6ab-421c-8405-faa2bad547af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:54.494851 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c9d6e0-c6ab-421c-8405-faa2bad547af admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:54.511419 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 933/1875] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:54 2022] PUT /identity/v3/projects/a6ea5cc4903842bc9c97a413a73a115b/users/bdfb7e5a13be4fdbb124b308647b823f/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:34:54.519614 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 943/1876] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:54.522524 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1493c0-9afd-4819-be85-fef5e71ad94e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:54.527234 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c50726-775c-4b6b-bf46-c805f535f4d0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:54.527637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c50726-775c-4b6b-bf46-c805f535f4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:54.527873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c50726-775c-4b6b-bf46-c805f535f4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:54.570281 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e3c50726-775c-4b6b-bf46-c805f535f4d0 None None] MFA Rules not processed for user `bdfb7e5a13be4fdbb124b308647b823f`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:54.572390 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e3c50726-775c-4b6b-bf46-c805f535f4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:54.645249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1493c0-9afd-4819-be85-fef5e71ad94e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:54.652372 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682da01-4ce8-489c-b5cb-608baad1b71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:54.653376 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682da01-4ce8-489c-b5cb-608baad1b71c admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:54.653666 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682da01-4ce8-489c-b5cb-608baad1b71c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:54.653978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682da01-4ce8-489c-b5cb-608baad1b71c admin admin] PATH_INFO: `/v3/projects/c82a54e10702439e919e14c5ed39ed01/users/a9d86993daf3432d916e2ac48d8535fc/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:54.671263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682da01-4ce8-489c-b5cb-608baad1b71c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c82a54e10702439e919e14c5ed39ed01, user_id=a9d86993daf3432d916e2ac48d8535fc, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:54.674899 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682da01-4ce8-489c-b5cb-608baad1b71c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:54.676585 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 944/1877] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:54 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:34:54.681124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c50726-775c-4b6b-bf46-c805f535f4d0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:54.694755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c50726-775c-4b6b-bf46-c805f535f4d0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:54.701528 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c28f0-af8b-442a-9bf9-34695725bc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:54.702233 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c28f0-af8b-442a-9bf9-34695725bc42 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:54.702514 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c28f0-af8b-442a-9bf9-34695725bc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:54.702789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c28f0-af8b-442a-9bf9-34695725bc42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:54.704280 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c28f0-af8b-442a-9bf9-34695725bc42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:54.708818 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c28f0-af8b-442a-9bf9-34695725bc42 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:54.775536 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 934/1878] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:54 2022] PUT /identity/v3/projects/c82a54e10702439e919e14c5ed39ed01/users/a9d86993daf3432d916e2ac48d8535fc/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:54.789407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de1131-4e7f-47a3-aaff-3eb954ba15de None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:54.794309 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de1131-4e7f-47a3-aaff-3eb954ba15de None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:54.794309 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de1131-4e7f-47a3-aaff-3eb954ba15de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:54.826866 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f6de1131-4e7f-47a3-aaff-3eb954ba15de None None] MFA Rules not processed for user `a9d86993daf3432d916e2ac48d8535fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:54.828211 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f6de1131-4e7f-47a3-aaff-3eb954ba15de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:54.858978 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 945/1879] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:34:54 2022] POST /identity/v3/projects => generated 357 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:34:54.865012 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c28f0-af8b-442a-9bf9-34695725bc42 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:54.988440 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 935/1880] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:34:54.993886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de1131-4e7f-47a3-aaff-3eb954ba15de None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:55.018278 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c28f0-af8b-442a-9bf9-34695725bc42 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:55.027259 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770364e-8e44-4f32-90fa-dc1ddc6d788d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:55.028620 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770364e-8e44-4f32-90fa-dc1ddc6d788d admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:55.029205 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770364e-8e44-4f32-90fa-dc1ddc6d788d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:55.029771 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770364e-8e44-4f32-90fa-dc1ddc6d788d admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:55.031185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770364e-8e44-4f32-90fa-dc1ddc6d788d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:55.035793 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770364e-8e44-4f32-90fa-dc1ddc6d788d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:55.108424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de1131-4e7f-47a3-aaff-3eb954ba15de None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:55.117116 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b32ea5-6322-42a9-b414-1354a40680a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:55.117116 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b32ea5-6322-42a9-b414-1354a40680a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:55.117116 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b32ea5-6322-42a9-b414-1354a40680a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:55.117116 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b32ea5-6322-42a9-b414-1354a40680a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:55.118005 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b32ea5-6322-42a9-b414-1354a40680a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:55.120245 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b32ea5-6322-42a9-b414-1354a40680a6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:55.151116 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 946/1881] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:34:54 2022] POST /identity/v3/users => generated 285 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:34:55.155465 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770364e-8e44-4f32-90fa-dc1ddc6d788d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:55.168310 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770364e-8e44-4f32-90fa-dc1ddc6d788d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:55.177201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa54fff-4913-499f-bafc-337dcde54f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:55.178441 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa54fff-4913-499f-bafc-337dcde54f08 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:55.178928 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa54fff-4913-499f-bafc-337dcde54f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:55.179408 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa54fff-4913-499f-bafc-337dcde54f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:55.180694 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa54fff-4913-499f-bafc-337dcde54f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:55.183755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa54fff-4913-499f-bafc-337dcde54f08 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:55.202126 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 947/1882] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:34:55.210724 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa54fff-4913-499f-bafc-337dcde54f08 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:55.226171 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:55.242764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa54fff-4913-499f-bafc-337dcde54f08 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:55.243580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 936/1883] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:34:54 2022] POST /identity/v3/projects => generated 383 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:55.254304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b32ea5-6322-42a9-b414-1354a40680a6 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:55.308679 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:55.385769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2ebb5-686d-44be-97d7-894b2be6f74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:55.387613 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2ebb5-686d-44be-97d7-894b2be6f74b admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:55.392753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2ebb5-686d-44be-97d7-894b2be6f74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:55.392753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2ebb5-686d-44be-97d7-894b2be6f74b admin admin] PATH_INFO: `/v3/projects/cdc4ba50a153400d94565f3dabfe58cf/users/bee42955f82846b5bd9b0dd5d0e15aaa/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:55.397857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b32ea5-6322-42a9-b414-1354a40680a6 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:55.405360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd84ce-7486-4f96-9e78-d51894fcd31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:55.406813 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd84ce-7486-4f96-9e78-d51894fcd31a admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:55.407319 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd84ce-7486-4f96-9e78-d51894fcd31a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:55.407319 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bd84ce-7486-4f96-9e78-d51894fcd31a admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:55.408828 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd84ce-7486-4f96-9e78-d51894fcd31a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:55.412620 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bd84ce-7486-4f96-9e78-d51894fcd31a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:55.413888 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b2ebb5-686d-44be-97d7-894b2be6f74b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdc4ba50a153400d94565f3dabfe58cf, user_id=bee42955f82846b5bd9b0dd5d0e15aaa, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:55.416826 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b2ebb5-686d-44be-97d7-894b2be6f74b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:55.556959 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 948/1884] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:55 2022] PUT /identity/v3/projects/cdc4ba50a153400d94565f3dabfe58cf/users/bee42955f82846b5bd9b0dd5d0e15aaa/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:34:55.558188 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 937/1885] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:34:55 2022] POST /identity/v3/users => generated 298 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:34:55.560965 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd84ce-7486-4f96-9e78-d51894fcd31a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:55.561226 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2ebb5-686d-44be-97d7-894b2be6f74b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:55.579286 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bd84ce-7486-4f96-9e78-d51894fcd31a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:55.692264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2ebb5-686d-44be-97d7-894b2be6f74b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:55.702172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7266e-9a27-42ab-8bbe-5b42887d9fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:55.703155 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7266e-9a27-42ab-8bbe-5b42887d9fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:55.703155 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7266e-9a27-42ab-8bbe-5b42887d9fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:55.703437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe7266e-9a27-42ab-8bbe-5b42887d9fa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:55.705770 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7266e-9a27-42ab-8bbe-5b42887d9fa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:55.707910 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe7266e-9a27-42ab-8bbe-5b42887d9fa1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:55.724907 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 949/1886] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:55.730694 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7266e-9a27-42ab-8bbe-5b42887d9fa1 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:55.744964 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a11bdf-abc5-4242-a05f-f1b6ff6451c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:55.746263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a11bdf-abc5-4242-a05f-f1b6ff6451c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:55.746709 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a11bdf-abc5-4242-a05f-f1b6ff6451c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:55.747146 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a11bdf-abc5-4242-a05f-f1b6ff6451c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:55.748117 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a11bdf-abc5-4242-a05f-f1b6ff6451c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:55.748242 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe7266e-9a27-42ab-8bbe-5b42887d9fa1 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:55.750182 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a11bdf-abc5-4242-a05f-f1b6ff6451c5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:55.755938 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3c386-5e4c-477f-ae02-66767948d71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:55.757008 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3c386-5e4c-477f-ae02-66767948d71a admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:55.757299 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3c386-5e4c-477f-ae02-66767948d71a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:55.757576 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3c386-5e4c-477f-ae02-66767948d71a admin admin] PATH_INFO: `/v3/projects/cdc4ba50a153400d94565f3dabfe58cf/users/bee42955f82846b5bd9b0dd5d0e15aaa/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:55.760674 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3c386-5e4c-477f-ae02-66767948d71a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdc4ba50a153400d94565f3dabfe58cf, user_id=bee42955f82846b5bd9b0dd5d0e15aaa, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:55.767069 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3c386-5e4c-477f-ae02-66767948d71a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:55.767879 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 938/1887] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:55.770683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a11bdf-abc5-4242-a05f-f1b6ff6451c5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:55.783818 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a11bdf-abc5-4242-a05f-f1b6ff6451c5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:55.786845 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 950/1888] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:55 2022] PUT /identity/v3/projects/cdc4ba50a153400d94565f3dabfe58cf/users/bee42955f82846b5bd9b0dd5d0e15aaa/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:55.817550 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf228e93-0d87-43ed-bfb9-41d1f0358f1c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:55.822901 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf228e93-0d87-43ed-bfb9-41d1f0358f1c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:55.823506 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf228e93-0d87-43ed-bfb9-41d1f0358f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:55.880284 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-bf228e93-0d87-43ed-bfb9-41d1f0358f1c None None] MFA Rules not processed for user `bee42955f82846b5bd9b0dd5d0e15aaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:55.881898 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bf228e93-0d87-43ed-bfb9-41d1f0358f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:55.918540 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31272315-f59c-474f-8466-0bfcfe828d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:55.919592 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31272315-f59c-474f-8466-0bfcfe828d3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:55.919882 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31272315-f59c-474f-8466-0bfcfe828d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:55.920164 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31272315-f59c-474f-8466-0bfcfe828d3e admin admin] PATH_INFO: `/v3/projects/6d94ebd92bb8438b87ffe920922c7902/users/f4513c327fd54624acd858204a12e32f/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:55.952463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31272315-f59c-474f-8466-0bfcfe828d3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d94ebd92bb8438b87ffe920922c7902, user_id=f4513c327fd54624acd858204a12e32f, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:55.970653 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31272315-f59c-474f-8466-0bfcfe828d3e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:56.008057 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 951/1889] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:55 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:56.029045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf228e93-0d87-43ed-bfb9-41d1f0358f1c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:56.030846 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bf228e93-0d87-43ed-bfb9-41d1f0358f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:56.073703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 939/1890] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:55 2022] PUT /identity/v3/projects/6d94ebd92bb8438b87ffe920922c7902/users/f4513c327fd54624acd858204a12e32f/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:34:56.079652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31272315-f59c-474f-8466-0bfcfe828d3e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:56.107277 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf228e93-0d87-43ed-bfb9-41d1f0358f1c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:56.121797 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:56.176272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31272315-f59c-474f-8466-0bfcfe828d3e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:56.188944 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea20eb-0026-4566-a3d9-5f71048ce0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:56.190879 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea20eb-0026-4566-a3d9-5f71048ce0a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:56.190879 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea20eb-0026-4566-a3d9-5f71048ce0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:56.191178 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea20eb-0026-4566-a3d9-5f71048ce0a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:56.192154 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea20eb-0026-4566-a3d9-5f71048ce0a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:56.194064 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea20eb-0026-4566-a3d9-5f71048ce0a5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:56.213090 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 940/1891] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:34:56.222330 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea20eb-0026-4566-a3d9-5f71048ce0a5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:56.223890 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a0354-78d4-449a-aa5b-c159391b51ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:56.225053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a0354-78d4-449a-aa5b-c159391b51ca service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:56.225594 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a0354-78d4-449a-aa5b-c159391b51ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:56.226056 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a0354-78d4-449a-aa5b-c159391b51ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:56.251266 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea20eb-0026-4566-a3d9-5f71048ce0a5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:56.262588 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b27d02-4456-4efc-ba56-256a023186d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:56.264549 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b27d02-4456-4efc-ba56-256a023186d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:56.265031 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b27d02-4456-4efc-ba56-256a023186d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:56.265531 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b27d02-4456-4efc-ba56-256a023186d1 admin admin] PATH_INFO: `/v3/projects/6d94ebd92bb8438b87ffe920922c7902/users/f4513c327fd54624acd858204a12e32f/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:56.270859 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b27d02-4456-4efc-ba56-256a023186d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d94ebd92bb8438b87ffe920922c7902, user_id=f4513c327fd54624acd858204a12e32f, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:56.276807 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b27d02-4456-4efc-ba56-256a023186d1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:56.311763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 941/1892] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:34:56 2022] PUT /identity/v3/projects/6d94ebd92bb8438b87ffe920922c7902/users/f4513c327fd54624acd858204a12e32f/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:34:56.324767 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80b434-b8ed-45f7-86cf-10463dd10a75 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:56.325771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80b434-b8ed-45f7-86cf-10463dd10a75 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:56.326568 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80b434-b8ed-45f7-86cf-10463dd10a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:56.339737 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:56.410853 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a0354-78d4-449a-aa5b-c159391b51ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:56.413194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a0354-78d4-449a-aa5b-c159391b51ca service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:56.415845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fe80b434-b8ed-45f7-86cf-10463dd10a75 None None] MFA Rules not processed for user `f4513c327fd54624acd858204a12e32f`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:34:56.418012 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fe80b434-b8ed-45f7-86cf-10463dd10a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:56.449675 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 952/1893] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:56 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:34:56.551116 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 942/1894] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:34:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:34:56.568632 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a0354-78d4-449a-aa5b-c159391b51ca service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:56.570253 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-df1a0354-78d4-449a-aa5b-c159391b51ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:34:56.592972 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a0354-78d4-449a-aa5b-c159391b51ca service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:56.673051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06e4c0-360d-4536-9ae3-ceec4b711f0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:56.673980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06e4c0-360d-4536-9ae3-ceec4b711f0f service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:56.674444 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06e4c0-360d-4536-9ae3-ceec4b711f0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:56.674923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06e4c0-360d-4536-9ae3-ceec4b711f0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:56.692045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06e4c0-360d-4536-9ae3-ceec4b711f0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:56.694764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06e4c0-360d-4536-9ae3-ceec4b711f0f service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:56.714116 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 953/1895] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:56 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:34:56.759158 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80b434-b8ed-45f7-86cf-10463dd10a75 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:56.778119 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80b434-b8ed-45f7-86cf-10463dd10a75 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:56.781112 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6e8c0-686f-4f44-b258-bcd0f3ef6d3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:56.782900 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6e8c0-686f-4f44-b258-bcd0f3ef6d3b service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:56.783806 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6e8c0-686f-4f44-b258-bcd0f3ef6d3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:56.784222 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6e8c0-686f-4f44-b258-bcd0f3ef6d3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:56.900070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06e4c0-360d-4536-9ae3-ceec4b711f0f service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:34:56.904260 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef6e8c0-686f-4f44-b258-bcd0f3ef6d3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:56.907878 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef6e8c0-686f-4f44-b258-bcd0f3ef6d3b service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:56.925134 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06e4c0-360d-4536-9ae3-ceec4b711f0f service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:34:56.931853 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f61c3-c1e9-4322-b799-eaa4eb210e84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:34:56.932192 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 943/1896] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:56 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:34:56.933187 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f61c3-c1e9-4322-b799-eaa4eb210e84 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:34:56.933665 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f61c3-c1e9-4322-b799-eaa4eb210e84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:34:56.934206 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f61c3-c1e9-4322-b799-eaa4eb210e84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:34:57.078930 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f61c3-c1e9-4322-b799-eaa4eb210e84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:34:57.082138 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f61c3-c1e9-4322-b799-eaa4eb210e84 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:34:57.102930 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 954/1897] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:34:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:34:57.393434 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:58.424443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:59.236116 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:34:59.737089 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:00.257239 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:00.264589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6e8c0-686f-4f44-b258-bcd0f3ef6d3b service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:00.273473 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6e8c0-686f-4f44-b258-bcd0f3ef6d3b service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:00.281747 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43cead6-3968-4d8f-b123-8e944b91f8ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:00.282897 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cead6-3968-4d8f-b123-8e944b91f8ff service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:00.283341 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cead6-3968-4d8f-b123-8e944b91f8ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:00.283985 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cead6-3968-4d8f-b123-8e944b91f8ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:00.286247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e43cead6-3968-4d8f-b123-8e944b91f8ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:00.371567 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43cead6-3968-4d8f-b123-8e944b91f8ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:00.373054 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43cead6-3968-4d8f-b123-8e944b91f8ff service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:00.389320 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 944/1898] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:35:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:35:00.760424 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:00.903173 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f61c3-c1e9-4322-b799-eaa4eb210e84 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:00.994854 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f61c3-c1e9-4322-b799-eaa4eb210e84 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:01.002189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6376cfdf-d9db-4f69-9b19-820ed448cc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:01.003235 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376cfdf-d9db-4f69-9b19-820ed448cc04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:01.003585 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376cfdf-d9db-4f69-9b19-820ed448cc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:01.003881 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376cfdf-d9db-4f69-9b19-820ed448cc04 admin admin] PATH_INFO: `/v3/users/a9d86993daf3432d916e2ac48d8535fc` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:01.005191 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6376cfdf-d9db-4f69-9b19-820ed448cc04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9d86993daf3432d916e2ac48d8535fc)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:01.007153 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6376cfdf-d9db-4f69-9b19-820ed448cc04 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:01.318889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:01.375136 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6376cfdf-d9db-4f69-9b19-820ed448cc04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9d86993daf3432d916e2ac48d8535fc'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:01.375948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6376cfdf-d9db-4f69-9b19-820ed448cc04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9d86993daf3432d916e2ac48d8535fc'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:01.386398 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6376cfdf-d9db-4f69-9b19-820ed448cc04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9d86993daf3432d916e2ac48d8535fc'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:01.464109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6376cfdf-d9db-4f69-9b19-820ed448cc04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9d86993daf3432d916e2ac48d8535fc'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:01.464791 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6376cfdf-d9db-4f69-9b19-820ed448cc04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9d86993daf3432d916e2ac48d8535fc'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:01.485252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 955/1899] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:35:00 2022] DELETE /identity/v3/users/a9d86993daf3432d916e2ac48d8535fc => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:01.490607 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ce447-ef3f-417c-998a-c97b3f8d440e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:01.490968 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ce447-ef3f-417c-998a-c97b3f8d440e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:01.491245 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ce447-ef3f-417c-998a-c97b3f8d440e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:01.514307 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-290ce447-ef3f-417c-998a-c97b3f8d440e None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:01.515811 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-290ce447-ef3f-417c-998a-c97b3f8d440e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:01.603694 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 945/1900] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:01.619979 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6376cfdf-d9db-4f69-9b19-820ed448cc04 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:01.619979 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6376cfdf-d9db-4f69-9b19-820ed448cc04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:01.684699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6376cfdf-d9db-4f69-9b19-820ed448cc04 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:01.688704 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520f467-8c2e-4d1f-b032-7c6dc080d235 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:01.690471 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520f467-8c2e-4d1f-b032-7c6dc080d235 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:01.690764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520f467-8c2e-4d1f-b032-7c6dc080d235 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:01.691046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520f467-8c2e-4d1f-b032-7c6dc080d235 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:01.702204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520f467-8c2e-4d1f-b032-7c6dc080d235 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:01.703783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520f467-8c2e-4d1f-b032-7c6dc080d235 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:01.716433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 956/1901] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:35:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:35:02.389325 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:02.426626 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ce447-ef3f-417c-998a-c97b3f8d440e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:02.428918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-290ce447-ef3f-417c-998a-c97b3f8d440e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:02.459137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ce447-ef3f-417c-998a-c97b3f8d440e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:02.465817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21351be9-609a-4db1-892c-11804a73d9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:02.466886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21351be9-609a-4db1-892c-11804a73d9f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:02.467353 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21351be9-609a-4db1-892c-11804a73d9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:02.467683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21351be9-609a-4db1-892c-11804a73d9f2 admin admin] PATH_INFO: `/v3/projects/c82a54e10702439e919e14c5ed39ed01` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:02.469443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21351be9-609a-4db1-892c-11804a73d9f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c82a54e10702439e919e14c5ed39ed01)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:02.472383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21351be9-609a-4db1-892c-11804a73d9f2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:02.640513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-21351be9-609a-4db1-892c-11804a73d9f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c82a54e10702439e919e14c5ed39ed01', 'request_id': 'req-21351be9-609a-4db1-892c-11804a73d9f2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:02.647646 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-21351be9-609a-4db1-892c-11804a73d9f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c82a54e10702439e919e14c5ed39ed01', 'request_id': 'req-21351be9-609a-4db1-892c-11804a73d9f2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:02.656171 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-21351be9-609a-4db1-892c-11804a73d9f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c82a54e10702439e919e14c5ed39ed01', 'request_id': 'req-21351be9-609a-4db1-892c-11804a73d9f2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:02.821000 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-21351be9-609a-4db1-892c-11804a73d9f2 admin admin] The token cache is being invalidate because project c82a54e10702439e919e14c5ed39ed01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:35:02.821641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-21351be9-609a-4db1-892c-11804a73d9f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:02.822757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-21351be9-609a-4db1-892c-11804a73d9f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:02.826109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 946/1902] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:35:02 2022] DELETE /identity/v3/projects/c82a54e10702439e919e14c5ed39ed01 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:35:02.830814 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520f467-8c2e-4d1f-b032-7c6dc080d235 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:02.844511 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520f467-8c2e-4d1f-b032-7c6dc080d235 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:02.942649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec35f43-7742-4987-b2f7-a1dc86e9b82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:02.943861 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec35f43-7742-4987-b2f7-a1dc86e9b82e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:02.944409 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec35f43-7742-4987-b2f7-a1dc86e9b82e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:02.944702 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec35f43-7742-4987-b2f7-a1dc86e9b82e admin admin] PATH_INFO: `/v3/users/f4513c327fd54624acd858204a12e32f` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:02.946360 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec35f43-7742-4987-b2f7-a1dc86e9b82e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4513c327fd54624acd858204a12e32f)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:02.948804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec35f43-7742-4987-b2f7-a1dc86e9b82e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:02.956836 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:03.303840 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-aec35f43-7742-4987-b2f7-a1dc86e9b82e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4513c327fd54624acd858204a12e32f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:03.305060 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-aec35f43-7742-4987-b2f7-a1dc86e9b82e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4513c327fd54624acd858204a12e32f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:03.320378 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-aec35f43-7742-4987-b2f7-a1dc86e9b82e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4513c327fd54624acd858204a12e32f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:03.426945 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-aec35f43-7742-4987-b2f7-a1dc86e9b82e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4513c327fd54624acd858204a12e32f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:03.430199 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-aec35f43-7742-4987-b2f7-a1dc86e9b82e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4513c327fd54624acd858204a12e32f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:03.455724 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 957/1903] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:35:02 2022] DELETE /identity/v3/users/f4513c327fd54624acd858204a12e32f => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:35:03.994701 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21351be9-609a-4db1-892c-11804a73d9f2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:04.000349 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-21351be9-609a-4db1-892c-11804a73d9f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:04.017464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21351be9-609a-4db1-892c-11804a73d9f2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:04.092177 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea94f6-1310-4d86-804d-cc79be8f6a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:04.092996 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea94f6-1310-4d86-804d-cc79be8f6a11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:04.093280 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea94f6-1310-4d86-804d-cc79be8f6a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:04.093557 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea94f6-1310-4d86-804d-cc79be8f6a11 admin admin] PATH_INFO: `/v3/projects/6d94ebd92bb8438b87ffe920922c7902` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:04.094851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea94f6-1310-4d86-804d-cc79be8f6a11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d94ebd92bb8438b87ffe920922c7902)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:04.097858 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea94f6-1310-4d86-804d-cc79be8f6a11 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:04.111117 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:04.231285 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-32ea94f6-1310-4d86-804d-cc79be8f6a11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d94ebd92bb8438b87ffe920922c7902', 'request_id': 'req-32ea94f6-1310-4d86-804d-cc79be8f6a11'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:04.238942 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-32ea94f6-1310-4d86-804d-cc79be8f6a11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d94ebd92bb8438b87ffe920922c7902', 'request_id': 'req-32ea94f6-1310-4d86-804d-cc79be8f6a11'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:04.245421 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-32ea94f6-1310-4d86-804d-cc79be8f6a11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d94ebd92bb8438b87ffe920922c7902', 'request_id': 'req-32ea94f6-1310-4d86-804d-cc79be8f6a11'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:04.356829 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-32ea94f6-1310-4d86-804d-cc79be8f6a11 admin admin] The token cache is being invalidate because project 6d94ebd92bb8438b87ffe920922c7902 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:35:04.357283 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-32ea94f6-1310-4d86-804d-cc79be8f6a11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:04.358419 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-32ea94f6-1310-4d86-804d-cc79be8f6a11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:04.361202 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 947/1904] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:35:03 2022] DELETE /identity/v3/projects/6d94ebd92bb8438b87ffe920922c7902 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:04.369100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcc9bb-eb6a-4b49-a449-2017104e9178 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:04.369392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcc9bb-eb6a-4b49-a449-2017104e9178 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:04.369676 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcc9bb-eb6a-4b49-a449-2017104e9178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:04.390525 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-09bcc9bb-eb6a-4b49-a449-2017104e9178 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:04.391953 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-09bcc9bb-eb6a-4b49-a449-2017104e9178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:04.462650 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 958/1905] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:04.471619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca982d-2d13-4ec8-ae0a-9448e7ddf373 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:04.472062 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca982d-2d13-4ec8-ae0a-9448e7ddf373 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:04.472662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca982d-2d13-4ec8-ae0a-9448e7ddf373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:04.499782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c1ca982d-2d13-4ec8-ae0a-9448e7ddf373 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:04.501926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c1ca982d-2d13-4ec8-ae0a-9448e7ddf373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:04.516218 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 948/1906] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:35:04.521210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bcc9bb-eb6a-4b49-a449-2017104e9178 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:04.536156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bcc9bb-eb6a-4b49-a449-2017104e9178 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:04.542817 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e7949-2df3-4c8a-812c-9ba460badd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:04.543557 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e7949-2df3-4c8a-812c-9ba460badd0b admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:04.543859 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e7949-2df3-4c8a-812c-9ba460badd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:04.544492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e7949-2df3-4c8a-812c-9ba460badd0b admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:04.545241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e7949-2df3-4c8a-812c-9ba460badd0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:35:04.545972 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e7949-2df3-4c8a-812c-9ba460badd0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:04.546076 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:04.547262 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e7949-2df3-4c8a-812c-9ba460badd0b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:04.564659 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 959/1907] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:35:04.569307 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca982d-2d13-4ec8-ae0a-9448e7ddf373 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:04.579516 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca982d-2d13-4ec8-ae0a-9448e7ddf373 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:04.585260 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4363c7-aed2-40b4-b320-0529b795e1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:04.586091 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4363c7-aed2-40b4-b320-0529b795e1df admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:04.586430 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4363c7-aed2-40b4-b320-0529b795e1df admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:04.586775 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4363c7-aed2-40b4-b320-0529b795e1df admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:04.587595 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4363c7-aed2-40b4-b320-0529b795e1df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:04.589607 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4363c7-aed2-40b4-b320-0529b795e1df admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:04.700828 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 949/1908] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:35:04 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:35:04.702827 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e7949-2df3-4c8a-812c-9ba460badd0b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:04.715024 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e7949-2df3-4c8a-812c-9ba460badd0b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:04.782113 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462874a4-8412-4d46-9104-3753ff1445e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:04.782792 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462874a4-8412-4d46-9104-3753ff1445e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:04.783149 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462874a4-8412-4d46-9104-3753ff1445e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:04.783548 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462874a4-8412-4d46-9104-3753ff1445e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:04.784675 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462874a4-8412-4d46-9104-3753ff1445e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:04.786620 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462874a4-8412-4d46-9104-3753ff1445e3 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:04.958674 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 960/1909] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:35:04 2022] POST /identity/v3/users => generated 287 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:35:04.960704 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4363c7-aed2-40b4-b320-0529b795e1df admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:04.979814 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4363c7-aed2-40b4-b320-0529b795e1df admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:04.992874 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bc43a-6449-45f6-8460-9feb9770ed2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:04.992874 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bc43a-6449-45f6-8460-9feb9770ed2f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:04.993354 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bc43a-6449-45f6-8460-9feb9770ed2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:04.995516 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353bc43a-6449-45f6-8460-9feb9770ed2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:04.995516 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353bc43a-6449-45f6-8460-9feb9770ed2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:04.998732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353bc43a-6449-45f6-8460-9feb9770ed2f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:05.028132 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462874a4-8412-4d46-9104-3753ff1445e3 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:05.028626 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 950/1910] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:35:05.044434 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462874a4-8412-4d46-9104-3753ff1445e3 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:05.050022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0e956-d8e3-4916-9cd7-9672c545a7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:05.050704 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0e956-d8e3-4916-9cd7-9672c545a7c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:05.050987 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0e956-d8e3-4916-9cd7-9672c545a7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:05.051262 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0e956-d8e3-4916-9cd7-9672c545a7c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:05.052175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb0e956-d8e3-4916-9cd7-9672c545a7c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:05.054061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb0e956-d8e3-4916-9cd7-9672c545a7c1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:05.066246 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 961/1911] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:35:05.072180 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bc43a-6449-45f6-8460-9feb9770ed2f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:05.090100 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353bc43a-6449-45f6-8460-9feb9770ed2f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:05.102473 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a1d3f-d0b1-4727-a991-79323ae3a37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:05.104399 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a1d3f-d0b1-4727-a991-79323ae3a37d admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:05.105907 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a1d3f-d0b1-4727-a991-79323ae3a37d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:05.107063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a1d3f-d0b1-4727-a991-79323ae3a37d admin admin] PATH_INFO: `/v3/projects/2b419f833aa041fb95ab0c976acc2514/users/c94f3d20717b4a24bb5c260bd8c60e27/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:05.141000 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a1d3f-d0b1-4727-a991-79323ae3a37d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b419f833aa041fb95ab0c976acc2514, user_id=c94f3d20717b4a24bb5c260bd8c60e27, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:05.146798 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a1d3f-d0b1-4727-a991-79323ae3a37d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:05.157207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:05.275077 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 951/1912] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:35:05 2022] PUT /identity/v3/projects/2b419f833aa041fb95ab0c976acc2514/users/c94f3d20717b4a24bb5c260bd8c60e27/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:05.278204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d2e5c-31c3-40ed-955b-c59a8cb680a4 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:05.278650 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d2e5c-31c3-40ed-955b-c59a8cb680a4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:05.279109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284d2e5c-31c3-40ed-955b-c59a8cb680a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:05.313420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-284d2e5c-31c3-40ed-955b-c59a8cb680a4 None None] MFA Rules not processed for user `c94f3d20717b4a24bb5c260bd8c60e27`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:05.315242 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-284d2e5c-31c3-40ed-955b-c59a8cb680a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:05.390908 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 962/1913] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:05.393832 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a1d3f-d0b1-4727-a991-79323ae3a37d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:05.466429 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a1d3f-d0b1-4727-a991-79323ae3a37d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:05.471949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eedbc5-eecf-40c2-87cc-735f695804f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:05.472627 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eedbc5-eecf-40c2-87cc-735f695804f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:05.472910 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eedbc5-eecf-40c2-87cc-735f695804f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:05.473183 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eedbc5-eecf-40c2-87cc-735f695804f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:05.473897 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eedbc5-eecf-40c2-87cc-735f695804f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:05.475901 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eedbc5-eecf-40c2-87cc-735f695804f1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:05.602712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 952/1914] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:35:05 2022] POST /identity/v3/projects => generated 373 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:35:05.610598 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d2e5c-31c3-40ed-955b-c59a8cb680a4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:05.629205 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284d2e5c-31c3-40ed-955b-c59a8cb680a4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:05.638005 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:05.699184 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3ec729-9dfb-42eb-90b3-98029d383c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:05.699869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ec729-9dfb-42eb-90b3-98029d383c31 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:05.700156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ec729-9dfb-42eb-90b3-98029d383c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:05.700499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3ec729-9dfb-42eb-90b3-98029d383c31 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:05.701218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3ec729-9dfb-42eb-90b3-98029d383c31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:05.703377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3ec729-9dfb-42eb-90b3-98029d383c31 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:05.829015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 963/1915] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:35:05 2022] POST /identity/v3/users => generated 293 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:35:05.835402 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eedbc5-eecf-40c2-87cc-735f695804f1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:05.847205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eedbc5-eecf-40c2-87cc-735f695804f1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:05.852390 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:05.853417 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0a1ac-5b08-4acf-95f0-f84ce3902009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:05.854372 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0a1ac-5b08-4acf-95f0-f84ce3902009 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:05.858737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0a1ac-5b08-4acf-95f0-f84ce3902009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:05.858737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0a1ac-5b08-4acf-95f0-f84ce3902009 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:05.858737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0a1ac-5b08-4acf-95f0-f84ce3902009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:05.858737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc0a1ac-5b08-4acf-95f0-f84ce3902009 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:05.872722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 953/1916] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:35:05.878084 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3ec729-9dfb-42eb-90b3-98029d383c31 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:05.889983 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3ec729-9dfb-42eb-90b3-98029d383c31 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:05.895956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c452a89-72e0-4d23-b091-83eb53611ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:05.896859 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c452a89-72e0-4d23-b091-83eb53611ac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:05.897207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c452a89-72e0-4d23-b091-83eb53611ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:05.897484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c452a89-72e0-4d23-b091-83eb53611ac8 admin admin] PATH_INFO: `/v3/projects/2299a56fba844b57bed0ef5805edc0b6/users/34af583695494fd988313cf3da49a108/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:05.915230 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c452a89-72e0-4d23-b091-83eb53611ac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2299a56fba844b57bed0ef5805edc0b6, user_id=34af583695494fd988313cf3da49a108, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:05.918211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c452a89-72e0-4d23-b091-83eb53611ac8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:06.040048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 964/1917] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:35:05 2022] PUT /identity/v3/projects/2299a56fba844b57bed0ef5805edc0b6/users/34af583695494fd988313cf3da49a108/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:35:06.044048 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0a1ac-5b08-4acf-95f0-f84ce3902009 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:06.058253 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0a1ac-5b08-4acf-95f0-f84ce3902009 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:06.125330 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d225427-48dc-4706-8ba7-e8e2f6d12aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:06.126039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d225427-48dc-4706-8ba7-e8e2f6d12aba admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:06.126347 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d225427-48dc-4706-8ba7-e8e2f6d12aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:06.126629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d225427-48dc-4706-8ba7-e8e2f6d12aba admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:06.127956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d225427-48dc-4706-8ba7-e8e2f6d12aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:06.129594 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d225427-48dc-4706-8ba7-e8e2f6d12aba admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:06.140817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 954/1918] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:35:06.146320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c452a89-72e0-4d23-b091-83eb53611ac8 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:06.157776 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c452a89-72e0-4d23-b091-83eb53611ac8 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:06.162916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf728d5-76a3-4980-a2dc-b7f33a148ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:06.163943 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf728d5-76a3-4980-a2dc-b7f33a148ba6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:06.164228 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf728d5-76a3-4980-a2dc-b7f33a148ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:06.164507 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf728d5-76a3-4980-a2dc-b7f33a148ba6 admin admin] PATH_INFO: `/v3/projects/2299a56fba844b57bed0ef5805edc0b6/users/34af583695494fd988313cf3da49a108/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:06.166956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf728d5-76a3-4980-a2dc-b7f33a148ba6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2299a56fba844b57bed0ef5805edc0b6, user_id=34af583695494fd988313cf3da49a108, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:06.169671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf728d5-76a3-4980-a2dc-b7f33a148ba6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:06.181825 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 965/1919] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:35:06 2022] PUT /identity/v3/projects/2299a56fba844b57bed0ef5805edc0b6/users/34af583695494fd988313cf3da49a108/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:06.194654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1faa91-e2af-4225-bed8-c272645a2c18 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:06.194968 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1faa91-e2af-4225-bed8-c272645a2c18 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:06.195280 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1faa91-e2af-4225-bed8-c272645a2c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:06.226743 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2b1faa91-e2af-4225-bed8-c272645a2c18 None None] MFA Rules not processed for user `34af583695494fd988313cf3da49a108`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:06.227993 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2b1faa91-e2af-4225-bed8-c272645a2c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:06.324876 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 955/1920] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:06.346839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf728d5-76a3-4980-a2dc-b7f33a148ba6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:06.348511 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cbf728d5-76a3-4980-a2dc-b7f33a148ba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:06.425789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf728d5-76a3-4980-a2dc-b7f33a148ba6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:06.428383 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093ae5dd-b7d4-4bca-96ac-fb567f22ba7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:06.429307 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ae5dd-b7d4-4bca-96ac-fb567f22ba7f service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:06.429796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ae5dd-b7d4-4bca-96ac-fb567f22ba7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:06.430269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ae5dd-b7d4-4bca-96ac-fb567f22ba7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:06.455137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:06.509513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093ae5dd-b7d4-4bca-96ac-fb567f22ba7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:06.511556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093ae5dd-b7d4-4bca-96ac-fb567f22ba7f service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:06.516880 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:06.524386 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 966/1921] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:35:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:35:06.625646 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:06.801554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:07.440682 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:07.452856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1faa91-e2af-4225-bed8-c272645a2c18 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:07.454492 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2b1faa91-e2af-4225-bed8-c272645a2c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:07.536167 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1faa91-e2af-4225-bed8-c272645a2c18 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:07.541415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a862b-f9f1-4388-840f-789f8e77efce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:07.545388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a862b-f9f1-4388-840f-789f8e77efce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:07.545388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a862b-f9f1-4388-840f-789f8e77efce admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:07.545388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a862b-f9f1-4388-840f-789f8e77efce admin admin] PATH_INFO: `/v3/users/24a7c6eef5284d1e86531b6a5dfe63e2` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:07.545388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a862b-f9f1-4388-840f-789f8e77efce admin admin] RBAC: Authorizing `identity:delete_user(user_id=24a7c6eef5284d1e86531b6a5dfe63e2)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:07.546517 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a862b-f9f1-4388-840f-789f8e77efce admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:07.667325 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:07.899972 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cb5a862b-f9f1-4388-840f-789f8e77efce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24a7c6eef5284d1e86531b6a5dfe63e2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:07.900941 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cb5a862b-f9f1-4388-840f-789f8e77efce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24a7c6eef5284d1e86531b6a5dfe63e2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:08.043322 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cb5a862b-f9f1-4388-840f-789f8e77efce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24a7c6eef5284d1e86531b6a5dfe63e2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:08.052362 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cb5a862b-f9f1-4388-840f-789f8e77efce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24a7c6eef5284d1e86531b6a5dfe63e2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:08.053433 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cb5a862b-f9f1-4388-840f-789f8e77efce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24a7c6eef5284d1e86531b6a5dfe63e2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:08.075267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 956/1922] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:35:07 2022] DELETE /identity/v3/users/24a7c6eef5284d1e86531b6a5dfe63e2 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:08.080036 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87fd04-1b0c-497d-8145-539860865846 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:08.080520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87fd04-1b0c-497d-8145-539860865846 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:08.080727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87fd04-1b0c-497d-8145-539860865846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:08.098725 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7b87fd04-1b0c-497d-8145-539860865846 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:08.100018 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7b87fd04-1b0c-497d-8145-539860865846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:08.168984 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 967/1923] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:08.179840 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a862b-f9f1-4388-840f-789f8e77efce admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:08.181751 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cb5a862b-f9f1-4388-840f-789f8e77efce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:08.259070 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a862b-f9f1-4388-840f-789f8e77efce admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:08.262929 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dbd3d5-eb76-4769-8cb2-69b4544beb74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:08.263767 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbd3d5-eb76-4769-8cb2-69b4544beb74 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:08.264053 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbd3d5-eb76-4769-8cb2-69b4544beb74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:08.264528 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbd3d5-eb76-4769-8cb2-69b4544beb74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:08.275640 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dbd3d5-eb76-4769-8cb2-69b4544beb74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:08.279033 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dbd3d5-eb76-4769-8cb2-69b4544beb74 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:08.297750 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 957/1924] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:35:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:35:08.423897 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:08.526165 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:08.860756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:08.925345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87fd04-1b0c-497d-8145-539860865846 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:08.927463 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7b87fd04-1b0c-497d-8145-539860865846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:08.966617 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87fd04-1b0c-497d-8145-539860865846 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:08.974005 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b50a7-b837-46b7-897d-5691c1a15329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:08.975029 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b50a7-b837-46b7-897d-5691c1a15329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:08.975390 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b50a7-b837-46b7-897d-5691c1a15329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:08.975778 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b50a7-b837-46b7-897d-5691c1a15329 admin admin] PATH_INFO: `/v3/projects/2ec97a5a23174830a41a272d53d6de5c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:08.977145 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b50a7-b837-46b7-897d-5691c1a15329 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ec97a5a23174830a41a272d53d6de5c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:08.979158 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b50a7-b837-46b7-897d-5691c1a15329 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:09.103773 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-118b50a7-b837-46b7-897d-5691c1a15329 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ec97a5a23174830a41a272d53d6de5c', 'request_id': 'req-118b50a7-b837-46b7-897d-5691c1a15329'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:09.209605 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-118b50a7-b837-46b7-897d-5691c1a15329 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ec97a5a23174830a41a272d53d6de5c', 'request_id': 'req-118b50a7-b837-46b7-897d-5691c1a15329'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:09.219190 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-118b50a7-b837-46b7-897d-5691c1a15329 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ec97a5a23174830a41a272d53d6de5c', 'request_id': 'req-118b50a7-b837-46b7-897d-5691c1a15329'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:09.242246 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-118b50a7-b837-46b7-897d-5691c1a15329 admin admin] The token cache is being invalidate because project 2ec97a5a23174830a41a272d53d6de5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:35:09.243165 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-118b50a7-b837-46b7-897d-5691c1a15329 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:09.245379 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-118b50a7-b837-46b7-897d-5691c1a15329 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:09.250550 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 968/1925] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:35:08 2022] DELETE /identity/v3/projects/2ec97a5a23174830a41a272d53d6de5c => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:09.264631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dbd3d5-eb76-4769-8cb2-69b4544beb74 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:09.274940 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dbd3d5-eb76-4769-8cb2-69b4544beb74 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:09.328695 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:09.363334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421529af-aec8-4a9c-b359-389a3eae2e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:09.364758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421529af-aec8-4a9c-b359-389a3eae2e90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:09.365071 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421529af-aec8-4a9c-b359-389a3eae2e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:09.365332 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421529af-aec8-4a9c-b359-389a3eae2e90 admin admin] PATH_INFO: `/v3/users/c5f4ac61ec774888a9c1a4ad963d0359` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:09.366964 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421529af-aec8-4a9c-b359-389a3eae2e90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5f4ac61ec774888a9c1a4ad963d0359)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:09.369442 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421529af-aec8-4a9c-b359-389a3eae2e90 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:09.635729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-421529af-aec8-4a9c-b359-389a3eae2e90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5f4ac61ec774888a9c1a4ad963d0359'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:09.636536 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-421529af-aec8-4a9c-b359-389a3eae2e90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5f4ac61ec774888a9c1a4ad963d0359'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:09.829568 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-421529af-aec8-4a9c-b359-389a3eae2e90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5f4ac61ec774888a9c1a4ad963d0359'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:09.838734 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-421529af-aec8-4a9c-b359-389a3eae2e90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5f4ac61ec774888a9c1a4ad963d0359'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:09.839651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-421529af-aec8-4a9c-b359-389a3eae2e90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5f4ac61ec774888a9c1a4ad963d0359'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:09.859570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 958/1926] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:35:09 2022] DELETE /identity/v3/users/c5f4ac61ec774888a9c1a4ad963d0359 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:09.901819 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b50a7-b837-46b7-897d-5691c1a15329 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:09.903816 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-118b50a7-b837-46b7-897d-5691c1a15329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:09.917985 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b50a7-b837-46b7-897d-5691c1a15329 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:09.958865 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:09.990102 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a07ff-7534-4676-a612-d88a1b75ec45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:09.991255 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a07ff-7534-4676-a612-d88a1b75ec45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:09.991591 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a07ff-7534-4676-a612-d88a1b75ec45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:09.991898 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a07ff-7534-4676-a612-d88a1b75ec45 admin admin] PATH_INFO: `/v3/users/c94f3d20717b4a24bb5c260bd8c60e27` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:09.993613 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a07ff-7534-4676-a612-d88a1b75ec45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c94f3d20717b4a24bb5c260bd8c60e27)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:09.995633 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a07ff-7534-4676-a612-d88a1b75ec45 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:10.274860 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d02a07ff-7534-4676-a612-d88a1b75ec45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c94f3d20717b4a24bb5c260bd8c60e27'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:10.275635 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d02a07ff-7534-4676-a612-d88a1b75ec45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c94f3d20717b4a24bb5c260bd8c60e27'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:10.283208 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d02a07ff-7534-4676-a612-d88a1b75ec45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c94f3d20717b4a24bb5c260bd8c60e27'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:10.414916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d02a07ff-7534-4676-a612-d88a1b75ec45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c94f3d20717b4a24bb5c260bd8c60e27'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:10.415609 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d02a07ff-7534-4676-a612-d88a1b75ec45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c94f3d20717b4a24bb5c260bd8c60e27'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:10.435140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 969/1927] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:35:09 2022] DELETE /identity/v3/users/c94f3d20717b4a24bb5c260bd8c60e27 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:10.442123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9748396-0572-4b30-bb5f-71977d1ad1d8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:10.442413 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9748396-0572-4b30-bb5f-71977d1ad1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:10.442692 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9748396-0572-4b30-bb5f-71977d1ad1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:10.463622 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d9748396-0572-4b30-bb5f-71977d1ad1d8 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:10.464864 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d9748396-0572-4b30-bb5f-71977d1ad1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:10.539124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 959/1928] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:10.549234 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a07ff-7534-4676-a612-d88a1b75ec45 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:10.550900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d02a07ff-7534-4676-a612-d88a1b75ec45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:10.561882 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:10.642385 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a07ff-7534-4676-a612-d88a1b75ec45 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:10.649074 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d1408-94e9-4a0a-9bae-46a3b5fedb58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:10.662948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d1408-94e9-4a0a-9bae-46a3b5fedb58 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:10.662948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d1408-94e9-4a0a-9bae-46a3b5fedb58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:10.662948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d1408-94e9-4a0a-9bae-46a3b5fedb58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:10.663507 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9748396-0572-4b30-bb5f-71977d1ad1d8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:10.663507 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d9748396-0572-4b30-bb5f-71977d1ad1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:10.669275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d1408-94e9-4a0a-9bae-46a3b5fedb58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:10.673044 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d1408-94e9-4a0a-9bae-46a3b5fedb58 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:10.679503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9748396-0572-4b30-bb5f-71977d1ad1d8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:10.692730 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36691c-2d3c-471b-b35b-e2d4653eb5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:10.693522 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36691c-2d3c-471b-b35b-e2d4653eb5b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:10.693810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36691c-2d3c-471b-b35b-e2d4653eb5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:10.697046 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36691c-2d3c-471b-b35b-e2d4653eb5b5 admin admin] PATH_INFO: `/v3/projects/1caa5b0935204192b4520d37909f5fcb` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:10.698763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36691c-2d3c-471b-b35b-e2d4653eb5b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1caa5b0935204192b4520d37909f5fcb)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:10.699148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 970/1929] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:35:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:35:10.701365 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36691c-2d3c-471b-b35b-e2d4653eb5b5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:10.896575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ab36691c-2d3c-471b-b35b-e2d4653eb5b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1caa5b0935204192b4520d37909f5fcb', 'request_id': 'req-ab36691c-2d3c-471b-b35b-e2d4653eb5b5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:10.903530 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ab36691c-2d3c-471b-b35b-e2d4653eb5b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1caa5b0935204192b4520d37909f5fcb', 'request_id': 'req-ab36691c-2d3c-471b-b35b-e2d4653eb5b5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:10.910604 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ab36691c-2d3c-471b-b35b-e2d4653eb5b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1caa5b0935204192b4520d37909f5fcb', 'request_id': 'req-ab36691c-2d3c-471b-b35b-e2d4653eb5b5'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:11.058512 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ab36691c-2d3c-471b-b35b-e2d4653eb5b5 admin admin] The token cache is being invalidate because project 1caa5b0935204192b4520d37909f5fcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:35:11.058813 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ab36691c-2d3c-471b-b35b-e2d4653eb5b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:11.059577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ab36691c-2d3c-471b-b35b-e2d4653eb5b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:11.061164 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 960/1930] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:35:10 2022] DELETE /identity/v3/projects/1caa5b0935204192b4520d37909f5fcb => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:35:11.068198 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f16a2-e907-4d4b-868f-97df223bb656 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:11.068750 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f16a2-e907-4d4b-868f-97df223bb656 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:11.069134 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f16a2-e907-4d4b-868f-97df223bb656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:11.081656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:11.123764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5e7f16a2-e907-4d4b-868f-97df223bb656 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:11.124600 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5e7f16a2-e907-4d4b-868f-97df223bb656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:11.210783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 971/1931] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:11.221160 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ed9d5-74fe-4454-b51f-103d9f054f58 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:11.221610 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ed9d5-74fe-4454-b51f-103d9f054f58 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:11.222025 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ed9d5-74fe-4454-b51f-103d9f054f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:11.246722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a74ed9d5-74fe-4454-b51f-103d9f054f58 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:11.248051 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a74ed9d5-74fe-4454-b51f-103d9f054f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:11.257524 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 961/1932] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:35:11.262060 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f16a2-e907-4d4b-868f-97df223bb656 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:11.276541 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f16a2-e907-4d4b-868f-97df223bb656 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:11.292476 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db0743-7239-4da1-bb98-974c899836bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:11.293188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db0743-7239-4da1-bb98-974c899836bd admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:11.293555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db0743-7239-4da1-bb98-974c899836bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:11.293913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db0743-7239-4da1-bb98-974c899836bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:11.294749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db0743-7239-4da1-bb98-974c899836bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:35:11.295146 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db0743-7239-4da1-bb98-974c899836bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:11.296932 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db0743-7239-4da1-bb98-974c899836bd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:11.312882 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 972/1933] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:35:11.318252 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ed9d5-74fe-4454-b51f-103d9f054f58 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:11.328248 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ed9d5-74fe-4454-b51f-103d9f054f58 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:11.333942 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a712b34-91ba-41fa-bb72-1514ca098b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:11.334619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a712b34-91ba-41fa-bb72-1514ca098b5f admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:11.334917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a712b34-91ba-41fa-bb72-1514ca098b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:11.335195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a712b34-91ba-41fa-bb72-1514ca098b5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:11.335950 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a712b34-91ba-41fa-bb72-1514ca098b5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:11.338050 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a712b34-91ba-41fa-bb72-1514ca098b5f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:11.489614 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 962/1934] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:35:11 2022] POST /identity/v3/projects => generated 375 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:35:11.494913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db0743-7239-4da1-bb98-974c899836bd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:11.508286 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db0743-7239-4da1-bb98-974c899836bd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:11.568896 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a712b34-91ba-41fa-bb72-1514ca098b5f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:11.570768 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5a712b34-91ba-41fa-bb72-1514ca098b5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:11.602390 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5d494-6b11-42ba-af62-77cffc3bcfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:11.603283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5d494-6b11-42ba-af62-77cffc3bcfb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:11.603616 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5d494-6b11-42ba-af62-77cffc3bcfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:11.603900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5d494-6b11-42ba-af62-77cffc3bcfb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:11.604702 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5d494-6b11-42ba-af62-77cffc3bcfb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:11.606703 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5d494-6b11-42ba-af62-77cffc3bcfb4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:11.725455 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:11.725758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a712b34-91ba-41fa-bb72-1514ca098b5f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:11.732295 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 973/1935] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:35:11 2022] POST /identity/v3/users => generated 288 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:35:11.736797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5d494-6b11-42ba-af62-77cffc3bcfb4 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:11.746499 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a610ee7-bf89-40a3-bab0-7a7fe8a5aaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:11.747590 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a610ee7-bf89-40a3-bab0-7a7fe8a5aaef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:11.747929 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a610ee7-bf89-40a3-bab0-7a7fe8a5aaef admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:11.749870 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a610ee7-bf89-40a3-bab0-7a7fe8a5aaef admin admin] PATH_INFO: `/v3/projects/2b419f833aa041fb95ab0c976acc2514` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:11.752629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a610ee7-bf89-40a3-bab0-7a7fe8a5aaef admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b419f833aa041fb95ab0c976acc2514)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:11.758473 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5d494-6b11-42ba-af62-77cffc3bcfb4 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:11.759231 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a610ee7-bf89-40a3-bab0-7a7fe8a5aaef admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:11.774449 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8bf3fe-8fae-4c9c-b57e-06b04e8fd90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:11.775142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bf3fe-8fae-4c9c-b57e-06b04e8fd90a admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:11.775537 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bf3fe-8fae-4c9c-b57e-06b04e8fd90a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:11.776033 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bf3fe-8fae-4c9c-b57e-06b04e8fd90a admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:11.777141 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8bf3fe-8fae-4c9c-b57e-06b04e8fd90a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:11.780730 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8bf3fe-8fae-4c9c-b57e-06b04e8fd90a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:11.796676 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 974/1936] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:35:11.798165 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8bf3fe-8fae-4c9c-b57e-06b04e8fd90a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:11.810650 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8bf3fe-8fae-4c9c-b57e-06b04e8fd90a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:11.816745 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11398f13-e3cc-4c8b-89d1-7a62775490ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:11.817558 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11398f13-e3cc-4c8b-89d1-7a62775490ba admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:11.820786 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11398f13-e3cc-4c8b-89d1-7a62775490ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:11.820786 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11398f13-e3cc-4c8b-89d1-7a62775490ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:11.820786 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11398f13-e3cc-4c8b-89d1-7a62775490ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:11.823699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11398f13-e3cc-4c8b-89d1-7a62775490ba admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:11.841313 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 975/1937] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:35:11.852647 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11398f13-e3cc-4c8b-89d1-7a62775490ba admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:11.866281 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11398f13-e3cc-4c8b-89d1-7a62775490ba admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:11.872026 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dceb45-d0ac-4dce-826c-59282534b04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:11.873115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dceb45-d0ac-4dce-826c-59282534b04c admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:11.873494 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dceb45-d0ac-4dce-826c-59282534b04c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:11.873864 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dceb45-d0ac-4dce-826c-59282534b04c admin admin] PATH_INFO: `/v3/projects/10f2fa56d4ac4ba2839736e58b55e8ff/users/76b0aa02f6af48e691291fcb4c0da5b3/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:11.905083 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dceb45-d0ac-4dce-826c-59282534b04c admin admin] RBAC: Authorizing `identity:create_grant(project_id=10f2fa56d4ac4ba2839736e58b55e8ff, user_id=76b0aa02f6af48e691291fcb4c0da5b3, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:11.911327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dceb45-d0ac-4dce-826c-59282534b04c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:12.070520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 976/1938] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:35:11 2022] PUT /identity/v3/projects/10f2fa56d4ac4ba2839736e58b55e8ff/users/76b0aa02f6af48e691291fcb4c0da5b3/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:35:12.085589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6a610ee7-bf89-40a3-bab0-7a7fe8a5aaef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b419f833aa041fb95ab0c976acc2514', 'request_id': 'req-6a610ee7-bf89-40a3-bab0-7a7fe8a5aaef'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:12.091367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666371a7-2e12-4748-84a4-91f42287267f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:12.091776 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666371a7-2e12-4748-84a4-91f42287267f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:12.092299 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666371a7-2e12-4748-84a4-91f42287267f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:12.108643 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6a610ee7-bf89-40a3-bab0-7a7fe8a5aaef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b419f833aa041fb95ab0c976acc2514', 'request_id': 'req-6a610ee7-bf89-40a3-bab0-7a7fe8a5aaef'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:12.123256 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6a610ee7-bf89-40a3-bab0-7a7fe8a5aaef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b419f833aa041fb95ab0c976acc2514', 'request_id': 'req-6a610ee7-bf89-40a3-bab0-7a7fe8a5aaef'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:12.133309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-666371a7-2e12-4748-84a4-91f42287267f None None] MFA Rules not processed for user `76b0aa02f6af48e691291fcb4c0da5b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:12.135039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-666371a7-2e12-4748-84a4-91f42287267f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:12.209998 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 977/1939] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:35:12.220907 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666371a7-2e12-4748-84a4-91f42287267f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:12.223205 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-666371a7-2e12-4748-84a4-91f42287267f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:12.248426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6a610ee7-bf89-40a3-bab0-7a7fe8a5aaef admin admin] The token cache is being invalidate because project 2b419f833aa041fb95ab0c976acc2514 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:35:12.248718 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6a610ee7-bf89-40a3-bab0-7a7fe8a5aaef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:12.249356 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-6a610ee7-bf89-40a3-bab0-7a7fe8a5aaef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:12.250959 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 963/1940] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:35:11 2022] DELETE /identity/v3/projects/2b419f833aa041fb95ab0c976acc2514 => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:35:12.257258 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a610ee7-bf89-40a3-bab0-7a7fe8a5aaef admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:12.259173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6a610ee7-bf89-40a3-bab0-7a7fe8a5aaef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:12.284929 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666371a7-2e12-4748-84a4-91f42287267f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:12.288932 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8adb63-3905-4fce-b001-ce7fbdd373b0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec41c84c9375437590c42621ca5b400a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:12.289701 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8adb63-3905-4fce-b001-ce7fbdd373b0 service glance] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:12.289985 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8adb63-3905-4fce-b001-ce7fbdd373b0 service glance] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:12.290260 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8adb63-3905-4fce-b001-ce7fbdd373b0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:12.292144 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cb8adb63-3905-4fce-b001-ce7fbdd373b0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:12.305666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8adb63-3905-4fce-b001-ce7fbdd373b0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:12.307195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8adb63-3905-4fce-b001-ce7fbdd373b0 service glance] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:12.317749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 978/1941] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:35:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:35:12.352768 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a610ee7-bf89-40a3-bab0-7a7fe8a5aaef admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:12.360574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c811f2-fd0e-4358-aa12-6ce4db35027a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:12.365588 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c811f2-fd0e-4358-aa12-6ce4db35027a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:12.365588 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c811f2-fd0e-4358-aa12-6ce4db35027a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:12.365588 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c811f2-fd0e-4358-aa12-6ce4db35027a admin admin] PATH_INFO: `/v3/users/34af583695494fd988313cf3da49a108` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:12.365588 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c811f2-fd0e-4358-aa12-6ce4db35027a admin admin] RBAC: Authorizing `identity:delete_user(user_id=34af583695494fd988313cf3da49a108)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:12.367114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c811f2-fd0e-4358-aa12-6ce4db35027a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:12.436267 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003696ab-fa93-4089-8334-05f9c7f64001 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:12.436692 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003696ab-fa93-4089-8334-05f9c7f64001 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:12.437002 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003696ab-fa93-4089-8334-05f9c7f64001 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:12.438471 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 979/1942] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 30 16:35:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 16:35:12.458758 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e1010-4ce1-4c7c-ac1e-cb9f3745a215 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:12.458758 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e1010-4ce1-4c7c-ac1e-cb9f3745a215 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:12.458758 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e1010-4ce1-4c7c-ac1e-cb9f3745a215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:12.483630 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-854e1010-4ce1-4c7c-ac1e-cb9f3745a215 None None] MFA Rules not processed for user `a932a9390fdd47f3adcf053a60250944`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:12.484869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-854e1010-4ce1-4c7c-ac1e-cb9f3745a215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:12.576058 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 980/1943] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 30 16:35:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:12.587530 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e1010-4ce1-4c7c-ac1e-cb9f3745a215 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:12.587530 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-854e1010-4ce1-4c7c-ac1e-cb9f3745a215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:12.599116 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e1010-4ce1-4c7c-ac1e-cb9f3745a215 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:12.604985 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675e937-e098-4ca6-b881-b4aa6283b19b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a932a9390fdd47f3adcf053a60250944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:12.605716 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675e937-e098-4ca6-b881-b4aa6283b19b service placement] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:12.606595 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675e937-e098-4ca6-b881-b4aa6283b19b service placement] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:12.606595 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675e937-e098-4ca6-b881-b4aa6283b19b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:12.617641 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675e937-e098-4ca6-b881-b4aa6283b19b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:12.623203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675e937-e098-4ca6-b881-b4aa6283b19b service placement] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:12.634115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 981/1944] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:35:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:35:12.682793 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675e937-e098-4ca6-b881-b4aa6283b19b service placement] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:12.684472 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a675e937-e098-4ca6-b881-b4aa6283b19b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:12.685520 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-53c811f2-fd0e-4358-aa12-6ce4db35027a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34af583695494fd988313cf3da49a108'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:12.686893 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-53c811f2-fd0e-4358-aa12-6ce4db35027a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34af583695494fd988313cf3da49a108'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:12.701380 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675e937-e098-4ca6-b881-b4aa6283b19b service placement] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:12.706249 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485376cc-2f2d-4339-8223-a9de1fd9a327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:12.711108 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485376cc-2f2d-4339-8223-a9de1fd9a327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:12.711108 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485376cc-2f2d-4339-8223-a9de1fd9a327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:12.711108 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485376cc-2f2d-4339-8223-a9de1fd9a327 admin admin] PATH_INFO: `/v3/users/5703438f487a45abbc5b68b77519534c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:12.711108 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485376cc-2f2d-4339-8223-a9de1fd9a327 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5703438f487a45abbc5b68b77519534c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:12.711701 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485376cc-2f2d-4339-8223-a9de1fd9a327 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:12.764032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-53c811f2-fd0e-4358-aa12-6ce4db35027a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34af583695494fd988313cf3da49a108'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:12.775448 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-53c811f2-fd0e-4358-aa12-6ce4db35027a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34af583695494fd988313cf3da49a108'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:12.776370 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-53c811f2-fd0e-4358-aa12-6ce4db35027a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34af583695494fd988313cf3da49a108'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:12.813225 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 964/1945] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:35:12 2022] DELETE /identity/v3/users/34af583695494fd988313cf3da49a108 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:12.891841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:13.070963 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-485376cc-2f2d-4339-8223-a9de1fd9a327 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5703438f487a45abbc5b68b77519534c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:13.071672 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-485376cc-2f2d-4339-8223-a9de1fd9a327 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5703438f487a45abbc5b68b77519534c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:13.079257 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-485376cc-2f2d-4339-8223-a9de1fd9a327 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5703438f487a45abbc5b68b77519534c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:13.192370 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-485376cc-2f2d-4339-8223-a9de1fd9a327 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5703438f487a45abbc5b68b77519534c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:13.193214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-485376cc-2f2d-4339-8223-a9de1fd9a327 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5703438f487a45abbc5b68b77519534c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:13.218650 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 982/1946] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:35:12 2022] DELETE /identity/v3/users/5703438f487a45abbc5b68b77519534c => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:13.295500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:13.476566 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c811f2-fd0e-4358-aa12-6ce4db35027a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:13.479142 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-53c811f2-fd0e-4358-aa12-6ce4db35027a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:13.541332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485376cc-2f2d-4339-8223-a9de1fd9a327 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:13.542554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-485376cc-2f2d-4339-8223-a9de1fd9a327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:13.587826 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c811f2-fd0e-4358-aa12-6ce4db35027a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:13.656197 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485376cc-2f2d-4339-8223-a9de1fd9a327 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:13.663154 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd85a8f-7f1b-4635-a447-026a38867fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:13.664434 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd85a8f-7f1b-4635-a447-026a38867fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:13.665020 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd85a8f-7f1b-4635-a447-026a38867fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:13.665888 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd85a8f-7f1b-4635-a447-026a38867fd9 admin admin] PATH_INFO: `/v3/projects/2da3dc7978d5414cba40e9e08dc39d94` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:13.667764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd85a8f-7f1b-4635-a447-026a38867fd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2da3dc7978d5414cba40e9e08dc39d94)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:13.670364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd85a8f-7f1b-4635-a447-026a38867fd9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:13.677247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884091d-fdf2-4622-b2ce-07b23d40c9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:13.678853 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884091d-fdf2-4622-b2ce-07b23d40c9a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:13.679152 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884091d-fdf2-4622-b2ce-07b23d40c9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:13.679433 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884091d-fdf2-4622-b2ce-07b23d40c9a4 admin admin] PATH_INFO: `/v3/projects/2299a56fba844b57bed0ef5805edc0b6` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:13.680792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d884091d-fdf2-4622-b2ce-07b23d40c9a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2299a56fba844b57bed0ef5805edc0b6)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:13.682806 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d884091d-fdf2-4622-b2ce-07b23d40c9a4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:13.855683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d884091d-fdf2-4622-b2ce-07b23d40c9a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2299a56fba844b57bed0ef5805edc0b6', 'request_id': 'req-d884091d-fdf2-4622-b2ce-07b23d40c9a4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:13.868032 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ebd85a8f-7f1b-4635-a447-026a38867fd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2da3dc7978d5414cba40e9e08dc39d94', 'request_id': 'req-ebd85a8f-7f1b-4635-a447-026a38867fd9'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:13.868701 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d884091d-fdf2-4622-b2ce-07b23d40c9a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2299a56fba844b57bed0ef5805edc0b6', 'request_id': 'req-d884091d-fdf2-4622-b2ce-07b23d40c9a4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:13.878093 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d884091d-fdf2-4622-b2ce-07b23d40c9a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2299a56fba844b57bed0ef5805edc0b6', 'request_id': 'req-d884091d-fdf2-4622-b2ce-07b23d40c9a4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:14.109674 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ebd85a8f-7f1b-4635-a447-026a38867fd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2da3dc7978d5414cba40e9e08dc39d94', 'request_id': 'req-ebd85a8f-7f1b-4635-a447-026a38867fd9'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:14.119731 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ebd85a8f-7f1b-4635-a447-026a38867fd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2da3dc7978d5414cba40e9e08dc39d94', 'request_id': 'req-ebd85a8f-7f1b-4635-a447-026a38867fd9'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:14.135739 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ebd85a8f-7f1b-4635-a447-026a38867fd9 admin admin] The token cache is being invalidate because project 2da3dc7978d5414cba40e9e08dc39d94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:35:14.136415 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ebd85a8f-7f1b-4635-a447-026a38867fd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:14.137336 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ebd85a8f-7f1b-4635-a447-026a38867fd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:14.139234 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 983/1947] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:35:13 2022] DELETE /identity/v3/projects/2da3dc7978d5414cba40e9e08dc39d94 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:14.145541 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b974e0-0f86-421c-b27b-47a2fa28f5d8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:14.146333 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b974e0-0f86-421c-b27b-47a2fa28f5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:14.147741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b974e0-0f86-421c-b27b-47a2fa28f5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:14.178573 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f9b974e0-0f86-421c-b27b-47a2fa28f5d8 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:14.180696 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f9b974e0-0f86-421c-b27b-47a2fa28f5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:14.232093 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d884091d-fdf2-4622-b2ce-07b23d40c9a4 admin admin] The token cache is being invalidate because project 2299a56fba844b57bed0ef5805edc0b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:35:14.232611 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d884091d-fdf2-4622-b2ce-07b23d40c9a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:14.233541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d884091d-fdf2-4622-b2ce-07b23d40c9a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:14.236299 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 965/1948] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:35:13 2022] DELETE /identity/v3/projects/2299a56fba844b57bed0ef5805edc0b6 => generated 0 bytes in 761 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:14.243582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f62c6-303c-469b-a75b-fa078ba8419b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:14.244224 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f62c6-303c-469b-a75b-fa078ba8419b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:14.244803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f62c6-303c-469b-a75b-fa078ba8419b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:14.267610 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-de4f62c6-303c-469b-a75b-fa078ba8419b None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:14.268917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-de4f62c6-303c-469b-a75b-fa078ba8419b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:14.274223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 984/1949] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:14.279172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 966/1950] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:35:14.290311 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051c93d-b236-48c1-a1e1-fcf19e4349e8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:14.290497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e5aef-8fe1-4027-9c1e-b84638943317 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:14.290685 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051c93d-b236-48c1-a1e1-fcf19e4349e8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:14.290943 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051c93d-b236-48c1-a1e1-fcf19e4349e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:14.291113 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e5aef-8fe1-4027-9c1e-b84638943317 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:14.291276 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e5aef-8fe1-4027-9c1e-b84638943317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:14.311542 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b051c93d-b236-48c1-a1e1-fcf19e4349e8 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:14.312842 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b051c93d-b236-48c1-a1e1-fcf19e4349e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:14.313516 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9e8e5aef-8fe1-4027-9c1e-b84638943317 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:14.314718 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9e8e5aef-8fe1-4027-9c1e-b84638943317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:14.324993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 985/1951] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:14.325192 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 967/1952] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:35:14.330875 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051c93d-b236-48c1-a1e1-fcf19e4349e8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:14.331250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e5aef-8fe1-4027-9c1e-b84638943317 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:14.346562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051c93d-b236-48c1-a1e1-fcf19e4349e8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:14.348420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e5aef-8fe1-4027-9c1e-b84638943317 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:14.353299 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049f00f-a8ae-44b4-aed6-b0965a0c42ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:14.354118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049f00f-a8ae-44b4-aed6-b0965a0c42ba admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:14.354221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104179b9-06aa-4bbe-a319-24aba8d22165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:14.354498 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049f00f-a8ae-44b4-aed6-b0965a0c42ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:14.354825 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049f00f-a8ae-44b4-aed6-b0965a0c42ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:14.354933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104179b9-06aa-4bbe-a319-24aba8d22165 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:14.355201 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104179b9-06aa-4bbe-a319-24aba8d22165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:14.355566 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104179b9-06aa-4bbe-a319-24aba8d22165 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:14.355939 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049f00f-a8ae-44b4-aed6-b0965a0c42ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:35:14.356510 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049f00f-a8ae-44b4-aed6-b0965a0c42ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:14.356908 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104179b9-06aa-4bbe-a319-24aba8d22165 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:35:14.357295 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104179b9-06aa-4bbe-a319-24aba8d22165 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:14.359524 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104179b9-06aa-4bbe-a319-24aba8d22165 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:14.359779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049f00f-a8ae-44b4-aed6-b0965a0c42ba admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:14.379518 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 986/1953] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:35:14.381310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 968/1954] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:35:14.385707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049f00f-a8ae-44b4-aed6-b0965a0c42ba admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:14.392257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104179b9-06aa-4bbe-a319-24aba8d22165 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:14.397582 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049f00f-a8ae-44b4-aed6-b0965a0c42ba admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:14.403052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f02f0-e7fe-45c6-ad09-feef048d6fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:14.404021 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f02f0-e7fe-45c6-ad09-feef048d6fbc admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:14.404301 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f02f0-e7fe-45c6-ad09-feef048d6fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:14.404582 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f02f0-e7fe-45c6-ad09-feef048d6fbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:14.405280 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f02f0-e7fe-45c6-ad09-feef048d6fbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:14.407275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f02f0-e7fe-45c6-ad09-feef048d6fbc admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:14.410281 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104179b9-06aa-4bbe-a319-24aba8d22165 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:14.420835 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546727e-857b-4e1f-8e39-7c0e43170d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:14.422713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546727e-857b-4e1f-8e39-7c0e43170d34 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:14.423248 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546727e-857b-4e1f-8e39-7c0e43170d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:14.427008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546727e-857b-4e1f-8e39-7c0e43170d34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:14.428690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c546727e-857b-4e1f-8e39-7c0e43170d34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:14.433306 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c546727e-857b-4e1f-8e39-7c0e43170d34 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:14.546298 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 987/1955] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:35:14 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:35:14.552938 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f02f0-e7fe-45c6-ad09-feef048d6fbc admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:14.608899 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 969/1956] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:35:14 2022] POST /identity/v3/projects => generated 365 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:35:14.615013 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546727e-857b-4e1f-8e39-7c0e43170d34 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:14.646168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f02f0-e7fe-45c6-ad09-feef048d6fbc admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:14.700896 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546727e-857b-4e1f-8e39-7c0e43170d34 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:14.707576 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4af166-0133-406a-8124-274c796d76b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:14.708697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4af166-0133-406a-8124-274c796d76b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:14.709126 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4af166-0133-406a-8124-274c796d76b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:14.709695 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4af166-0133-406a-8124-274c796d76b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:14.710730 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4af166-0133-406a-8124-274c796d76b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:14.714291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4af166-0133-406a-8124-274c796d76b3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:14.732617 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187b8c9-a63d-44ea-94d6-af84225a3711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:14.733468 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187b8c9-a63d-44ea-94d6-af84225a3711 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:14.733694 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187b8c9-a63d-44ea-94d6-af84225a3711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:14.733970 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187b8c9-a63d-44ea-94d6-af84225a3711 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:14.734651 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187b8c9-a63d-44ea-94d6-af84225a3711 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:14.736773 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3187b8c9-a63d-44ea-94d6-af84225a3711 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:14.802133 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:14.904649 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 988/1957] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:35:14 2022] POST /identity/v3/users => generated 288 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:35:14.909546 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 970/1958] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:35:14 2022] POST /identity/v3/users => generated 283 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:35:14.914357 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4af166-0133-406a-8124-274c796d76b3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:14.914555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187b8c9-a63d-44ea-94d6-af84225a3711 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:14.929928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4af166-0133-406a-8124-274c796d76b3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:14.935638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010e1ff-b330-487c-9832-c637f72913df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:14.939372 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187b8c9-a63d-44ea-94d6-af84225a3711 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:14.939540 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010e1ff-b330-487c-9832-c637f72913df admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:14.939540 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010e1ff-b330-487c-9832-c637f72913df admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:14.939540 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010e1ff-b330-487c-9832-c637f72913df admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:14.939540 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010e1ff-b330-487c-9832-c637f72913df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:14.942372 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457eca2-24e0-4061-a92e-ae31db51408d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:14.942547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010e1ff-b330-487c-9832-c637f72913df admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:14.943162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457eca2-24e0-4061-a92e-ae31db51408d admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:14.943551 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457eca2-24e0-4061-a92e-ae31db51408d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:14.944038 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457eca2-24e0-4061-a92e-ae31db51408d admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:14.945196 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457eca2-24e0-4061-a92e-ae31db51408d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:14.947090 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457eca2-24e0-4061-a92e-ae31db51408d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:14.959112 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 971/1959] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:35:14.963632 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 989/1960] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:35:14.965453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010e1ff-b330-487c-9832-c637f72913df admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:14.971557 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457eca2-24e0-4061-a92e-ae31db51408d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:14.988814 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010e1ff-b330-487c-9832-c637f72913df admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:14.992586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457eca2-24e0-4061-a92e-ae31db51408d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:14.996868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb8a29-4948-4cd4-9a13-41e0a1a24b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:14.997789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb8a29-4948-4cd4-9a13-41e0a1a24b1e admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:14.998187 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb8a29-4948-4cd4-9a13-41e0a1a24b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:14.998654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb8a29-4948-4cd4-9a13-41e0a1a24b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:15.000164 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb8a29-4948-4cd4-9a13-41e0a1a24b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:15.002390 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb8a29-4948-4cd4-9a13-41e0a1a24b1e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:15.015391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a2484-5c5f-4aed-a517-6b1e80f894b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:15.016994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a2484-5c5f-4aed-a517-6b1e80f894b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:15.017460 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a2484-5c5f-4aed-a517-6b1e80f894b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:15.019069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 972/1961] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:35:15.019750 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a2484-5c5f-4aed-a517-6b1e80f894b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:15.020432 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a2484-5c5f-4aed-a517-6b1e80f894b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:15.024152 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb8a29-4948-4cd4-9a13-41e0a1a24b1e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:15.025761 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a2484-5c5f-4aed-a517-6b1e80f894b2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:15.037576 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb8a29-4948-4cd4-9a13-41e0a1a24b1e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:15.043245 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 990/1962] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:35:15.044209 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d2b2d5-d891-4e68-bf6c-d5d58913818e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:15.047762 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2b2d5-d891-4e68-bf6c-d5d58913818e admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:15.047762 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2b2d5-d891-4e68-bf6c-d5d58913818e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:15.047762 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d2b2d5-d891-4e68-bf6c-d5d58913818e admin admin] PATH_INFO: `/v3/projects/28f8719c5d4946fd87ba824631d5b580/users/0973888f233c43e3bf2af18ebd01a932/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:15.052672 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a2484-5c5f-4aed-a517-6b1e80f894b2 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:15.065462 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a2484-5c5f-4aed-a517-6b1e80f894b2 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:15.070582 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f2daa-2bc1-4159-b875-2df7eafe11d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:15.071371 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d2b2d5-d891-4e68-bf6c-d5d58913818e admin admin] RBAC: Authorizing `identity:create_grant(project_id=28f8719c5d4946fd87ba824631d5b580, user_id=0973888f233c43e3bf2af18ebd01a932, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:15.071670 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f2daa-2bc1-4159-b875-2df7eafe11d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:15.072066 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f2daa-2bc1-4159-b875-2df7eafe11d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:15.072388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f2daa-2bc1-4159-b875-2df7eafe11d4 admin admin] PATH_INFO: `/v3/projects/ecc2efc61aad4698aaf73cb01b98bf08/users/939467dd5f9f420ebd37665d6afae392/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:15.074447 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d2b2d5-d891-4e68-bf6c-d5d58913818e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:15.090009 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f2daa-2bc1-4159-b875-2df7eafe11d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecc2efc61aad4698aaf73cb01b98bf08, user_id=939467dd5f9f420ebd37665d6afae392, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:15.092786 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f2daa-2bc1-4159-b875-2df7eafe11d4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:15.150498 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:15.183608 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 991/1963] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:35:15 2022] PUT /identity/v3/projects/ecc2efc61aad4698aaf73cb01b98bf08/users/939467dd5f9f420ebd37665d6afae392/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:35:15.183816 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 973/1964] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:35:15 2022] PUT /identity/v3/projects/28f8719c5d4946fd87ba824631d5b580/users/0973888f233c43e3bf2af18ebd01a932/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:35:15.194685 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71cab3-638c-4030-807a-b648ae4ae66c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:15.194974 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71cab3-638c-4030-807a-b648ae4ae66c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:15.195253 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71cab3-638c-4030-807a-b648ae4ae66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:15.196896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9eb90-0c97-4f9c-98f1-26c2a6ab0a74 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:15.197215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9eb90-0c97-4f9c-98f1-26c2a6ab0a74 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:15.197602 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9eb90-0c97-4f9c-98f1-26c2a6ab0a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:15.223782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5a71cab3-638c-4030-807a-b648ae4ae66c None None] MFA Rules not processed for user `939467dd5f9f420ebd37665d6afae392`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:15.225086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5a71cab3-638c-4030-807a-b648ae4ae66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:15.225478 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-59f9eb90-0c97-4f9c-98f1-26c2a6ab0a74 None None] MFA Rules not processed for user `0973888f233c43e3bf2af18ebd01a932`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:15.227052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-59f9eb90-0c97-4f9c-98f1-26c2a6ab0a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:15.307889 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 992/1965] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:35:15.319594 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 974/1966] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:15 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:15.321589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9eb90-0c97-4f9c-98f1-26c2a6ab0a74 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:15.324068 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-59f9eb90-0c97-4f9c-98f1-26c2a6ab0a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:15.326989 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a71cab3-638c-4030-807a-b648ae4ae66c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:15.328351 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5a71cab3-638c-4030-807a-b648ae4ae66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:15.391557 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a71cab3-638c-4030-807a-b648ae4ae66c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:15.393018 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9eb90-0c97-4f9c-98f1-26c2a6ab0a74 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:15.397165 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca71c2-7167-43b7-999e-5cea1ed575a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:15.397846 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca71c2-7167-43b7-999e-5cea1ed575a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:15.397983 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2648c3db-e00d-4558-9429-bbd39beb2e8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:15.398170 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca71c2-7167-43b7-999e-5cea1ed575a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:15.398443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca71c2-7167-43b7-999e-5cea1ed575a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:15.398654 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648c3db-e00d-4558-9429-bbd39beb2e8e service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:15.398941 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648c3db-e00d-4558-9429-bbd39beb2e8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:15.399271 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2648c3db-e00d-4558-9429-bbd39beb2e8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:15.409058 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2648c3db-e00d-4558-9429-bbd39beb2e8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:15.409439 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca71c2-7167-43b7-999e-5cea1ed575a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:15.411328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca71c2-7167-43b7-999e-5cea1ed575a6 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:15.411744 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2648c3db-e00d-4558-9429-bbd39beb2e8e service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:15.423422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 975/1967] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:35:15 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:35:15.427947 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 993/1968] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:35:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:35:15.627164 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:16.047212 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:16.815730 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:17.130462 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:18.160873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:18.801441 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:23.756936 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca71c2-7167-43b7-999e-5cea1ed575a6 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:23.759100 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4aca71c2-7167-43b7-999e-5cea1ed575a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:23.768910 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:23.838599 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca71c2-7167-43b7-999e-5cea1ed575a6 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:23.844001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0e4a7-1a67-497d-855d-52611c62a12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:23.845109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0e4a7-1a67-497d-855d-52611c62a12a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:23.845556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0e4a7-1a67-497d-855d-52611c62a12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:23.845904 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0e4a7-1a67-497d-855d-52611c62a12a admin admin] PATH_INFO: `/v3/users/bdfb7e5a13be4fdbb124b308647b823f` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:23.847757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0e4a7-1a67-497d-855d-52611c62a12a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdfb7e5a13be4fdbb124b308647b823f)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:23.850390 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a0e4a7-1a67-497d-855d-52611c62a12a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:24.097827 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-23a0e4a7-1a67-497d-855d-52611c62a12a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdfb7e5a13be4fdbb124b308647b823f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:24.098405 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-23a0e4a7-1a67-497d-855d-52611c62a12a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdfb7e5a13be4fdbb124b308647b823f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:24.179870 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-23a0e4a7-1a67-497d-855d-52611c62a12a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdfb7e5a13be4fdbb124b308647b823f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:24.187910 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-23a0e4a7-1a67-497d-855d-52611c62a12a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdfb7e5a13be4fdbb124b308647b823f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:24.188714 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-23a0e4a7-1a67-497d-855d-52611c62a12a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdfb7e5a13be4fdbb124b308647b823f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:24.209313 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 976/1969] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:35:23 2022] DELETE /identity/v3/users/bdfb7e5a13be4fdbb124b308647b823f => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:24.216193 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d16a1-45a3-44ad-8564-1e879590cae0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:24.216775 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d16a1-45a3-44ad-8564-1e879590cae0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:24.217127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d16a1-45a3-44ad-8564-1e879590cae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:24.240460 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f22d16a1-45a3-44ad-8564-1e879590cae0 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:24.242654 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f22d16a1-45a3-44ad-8564-1e879590cae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:24.312868 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 994/1970] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:24.323170 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0e4a7-1a67-497d-855d-52611c62a12a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:24.324651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-23a0e4a7-1a67-497d-855d-52611c62a12a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:24.402134 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0e4a7-1a67-497d-855d-52611c62a12a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:24.407283 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5b90c-7ea1-4f20-a55e-abf82c405fa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:24.408498 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5b90c-7ea1-4f20-a55e-abf82c405fa8 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:24.409170 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5b90c-7ea1-4f20-a55e-abf82c405fa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:24.409649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5b90c-7ea1-4f20-a55e-abf82c405fa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:24.436227 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de5b90c-7ea1-4f20-a55e-abf82c405fa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:24.438183 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de5b90c-7ea1-4f20-a55e-abf82c405fa8 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:24.450447 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 977/1971] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:35:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:35:25.055896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d16a1-45a3-44ad-8564-1e879590cae0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:25.057386 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f22d16a1-45a3-44ad-8564-1e879590cae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:25.070219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d16a1-45a3-44ad-8564-1e879590cae0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:25.076461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f467817-b30d-4c6b-bd08-9940b64ef40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:25.079312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f467817-b30d-4c6b-bd08-9940b64ef40c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:25.079312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f467817-b30d-4c6b-bd08-9940b64ef40c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:25.079312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f467817-b30d-4c6b-bd08-9940b64ef40c admin admin] PATH_INFO: `/v3/projects/a6ea5cc4903842bc9c97a413a73a115b` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:25.079312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f467817-b30d-4c6b-bd08-9940b64ef40c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6ea5cc4903842bc9c97a413a73a115b)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:25.081219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f467817-b30d-4c6b-bd08-9940b64ef40c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:25.207413 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0f467817-b30d-4c6b-bd08-9940b64ef40c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6ea5cc4903842bc9c97a413a73a115b', 'request_id': 'req-0f467817-b30d-4c6b-bd08-9940b64ef40c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:25.302759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0f467817-b30d-4c6b-bd08-9940b64ef40c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6ea5cc4903842bc9c97a413a73a115b', 'request_id': 'req-0f467817-b30d-4c6b-bd08-9940b64ef40c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:25.311021 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0f467817-b30d-4c6b-bd08-9940b64ef40c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6ea5cc4903842bc9c97a413a73a115b', 'request_id': 'req-0f467817-b30d-4c6b-bd08-9940b64ef40c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:25.325722 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0f467817-b30d-4c6b-bd08-9940b64ef40c admin admin] The token cache is being invalidate because project a6ea5cc4903842bc9c97a413a73a115b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:35:25.326016 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0f467817-b30d-4c6b-bd08-9940b64ef40c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:25.326751 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0f467817-b30d-4c6b-bd08-9940b64ef40c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:25.328609 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 995/1972] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:35:25 2022] DELETE /identity/v3/projects/a6ea5cc4903842bc9c97a413a73a115b => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:25.332840 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5b90c-7ea1-4f20-a55e-abf82c405fa8 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:25.343953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5b90c-7ea1-4f20-a55e-abf82c405fa8 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:25.374303 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:25.427150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc102e42-68de-4df3-85f1-0bc0de596b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:25.428101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc102e42-68de-4df3-85f1-0bc0de596b49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:25.428405 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc102e42-68de-4df3-85f1-0bc0de596b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:25.428676 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc102e42-68de-4df3-85f1-0bc0de596b49 admin admin] PATH_INFO: `/v3/users/bee42955f82846b5bd9b0dd5d0e15aaa` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:25.431016 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc102e42-68de-4df3-85f1-0bc0de596b49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bee42955f82846b5bd9b0dd5d0e15aaa)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:25.440895 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc102e42-68de-4df3-85f1-0bc0de596b49 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:25.765222 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dc102e42-68de-4df3-85f1-0bc0de596b49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bee42955f82846b5bd9b0dd5d0e15aaa'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:25.766043 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dc102e42-68de-4df3-85f1-0bc0de596b49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bee42955f82846b5bd9b0dd5d0e15aaa'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:25.894032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dc102e42-68de-4df3-85f1-0bc0de596b49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bee42955f82846b5bd9b0dd5d0e15aaa'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:25.905884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dc102e42-68de-4df3-85f1-0bc0de596b49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bee42955f82846b5bd9b0dd5d0e15aaa'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:25.910726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dc102e42-68de-4df3-85f1-0bc0de596b49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bee42955f82846b5bd9b0dd5d0e15aaa'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:25.936516 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 978/1973] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:35:25 2022] DELETE /identity/v3/users/bee42955f82846b5bd9b0dd5d0e15aaa => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:26.673079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f467817-b30d-4c6b-bd08-9940b64ef40c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:26.676222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0f467817-b30d-4c6b-bd08-9940b64ef40c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:26.689161 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f467817-b30d-4c6b-bd08-9940b64ef40c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:26.770815 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb0145e-0024-4ad1-903b-be7087ddf6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:26.771735 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0145e-0024-4ad1-903b-be7087ddf6ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:26.772018 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0145e-0024-4ad1-903b-be7087ddf6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:26.772389 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0145e-0024-4ad1-903b-be7087ddf6ac admin admin] PATH_INFO: `/v3/projects/cdc4ba50a153400d94565f3dabfe58cf` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:26.774230 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb0145e-0024-4ad1-903b-be7087ddf6ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdc4ba50a153400d94565f3dabfe58cf)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:26.776230 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb0145e-0024-4ad1-903b-be7087ddf6ac admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:26.803770 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:26.983306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bbb0145e-0024-4ad1-903b-be7087ddf6ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdc4ba50a153400d94565f3dabfe58cf', 'request_id': 'req-bbb0145e-0024-4ad1-903b-be7087ddf6ac'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:27.125628 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bbb0145e-0024-4ad1-903b-be7087ddf6ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdc4ba50a153400d94565f3dabfe58cf', 'request_id': 'req-bbb0145e-0024-4ad1-903b-be7087ddf6ac'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:27.132832 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bbb0145e-0024-4ad1-903b-be7087ddf6ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdc4ba50a153400d94565f3dabfe58cf', 'request_id': 'req-bbb0145e-0024-4ad1-903b-be7087ddf6ac'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:27.143958 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bbb0145e-0024-4ad1-903b-be7087ddf6ac admin admin] The token cache is being invalidate because project cdc4ba50a153400d94565f3dabfe58cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:35:27.144257 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bbb0145e-0024-4ad1-903b-be7087ddf6ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:27.145035 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bbb0145e-0024-4ad1-903b-be7087ddf6ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:27.146614 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 996/1974] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:35:26 2022] DELETE /identity/v3/projects/cdc4ba50a153400d94565f3dabfe58cf => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:35:27.152384 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac510d-0fec-4644-859a-da08cd45f678 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:27.152936 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac510d-0fec-4644-859a-da08cd45f678 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:27.152936 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac510d-0fec-4644-859a-da08cd45f678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:27.172974 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-34ac510d-0fec-4644-859a-da08cd45f678 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:27.184147 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-34ac510d-0fec-4644-859a-da08cd45f678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:27.244587 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 979/1975] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:27.252610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd669eb-c22a-4846-a73d-4927530df2e7 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:27.253118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd669eb-c22a-4846-a73d-4927530df2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:27.253996 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd669eb-c22a-4846-a73d-4927530df2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:27.280331 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ebd669eb-c22a-4846-a73d-4927530df2e7 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:27.282430 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ebd669eb-c22a-4846-a73d-4927530df2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:27.296830 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 997/1976] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:35:27.301180 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac510d-0fec-4644-859a-da08cd45f678 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:27.327744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac510d-0fec-4644-859a-da08cd45f678 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:27.327744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e256794-05d1-4c92-bab4-24fdddb26a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:27.327744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e256794-05d1-4c92-bab4-24fdddb26a50 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:27.327744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e256794-05d1-4c92-bab4-24fdddb26a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:27.328160 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e256794-05d1-4c92-bab4-24fdddb26a50 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:27.328160 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e256794-05d1-4c92-bab4-24fdddb26a50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:35:27.328160 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e256794-05d1-4c92-bab4-24fdddb26a50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:27.328160 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e256794-05d1-4c92-bab4-24fdddb26a50 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:27.338371 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 980/1977] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:35:27.343069 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd669eb-c22a-4846-a73d-4927530df2e7 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:27.354055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd669eb-c22a-4846-a73d-4927530df2e7 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:27.359391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbb718-4415-43e3-a182-56768d4471f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:27.360086 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbb718-4415-43e3-a182-56768d4471f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:27.360373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbb718-4415-43e3-a182-56768d4471f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:27.360654 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbb718-4415-43e3-a182-56768d4471f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:27.361360 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbb718-4415-43e3-a182-56768d4471f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:27.363276 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbb718-4415-43e3-a182-56768d4471f9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:27.514352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 998/1978] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:35:27 2022] POST /identity/v3/projects => generated 377 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:35:27.519179 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e256794-05d1-4c92-bab4-24fdddb26a50 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:27.531503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e256794-05d1-4c92-bab4-24fdddb26a50 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:27.580656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:27.617547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849a0ce-ebcc-4f62-a9c9-dc8ba47f80ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:27.618233 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849a0ce-ebcc-4f62-a9c9-dc8ba47f80ff admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:27.618519 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849a0ce-ebcc-4f62-a9c9-dc8ba47f80ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:27.618832 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849a0ce-ebcc-4f62-a9c9-dc8ba47f80ff admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:27.619661 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849a0ce-ebcc-4f62-a9c9-dc8ba47f80ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:27.621576 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849a0ce-ebcc-4f62-a9c9-dc8ba47f80ff admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:27.741252 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 981/1979] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:35:27 2022] POST /identity/v3/users => generated 289 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:35:27.745301 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbb718-4415-43e3-a182-56768d4471f9 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:27.759020 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbb718-4415-43e3-a182-56768d4471f9 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:27.764731 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c03a06-fa25-4725-979b-db732bb7d851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:27.765504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c03a06-fa25-4725-979b-db732bb7d851 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:27.765916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c03a06-fa25-4725-979b-db732bb7d851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:27.766300 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c03a06-fa25-4725-979b-db732bb7d851 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:27.767144 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c03a06-fa25-4725-979b-db732bb7d851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:27.769594 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c03a06-fa25-4725-979b-db732bb7d851 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:27.787239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 999/1980] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:35:27.795427 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849a0ce-ebcc-4f62-a9c9-dc8ba47f80ff admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:27.806237 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849a0ce-ebcc-4f62-a9c9-dc8ba47f80ff admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:27.810939 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ee743-ec7d-4626-90ca-0457a6c8ca0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:27.811641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ee743-ec7d-4626-90ca-0457a6c8ca0f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:27.811928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ee743-ec7d-4626-90ca-0457a6c8ca0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:27.812251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ee743-ec7d-4626-90ca-0457a6c8ca0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:27.813027 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ee743-ec7d-4626-90ca-0457a6c8ca0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:27.814639 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ee743-ec7d-4626-90ca-0457a6c8ca0f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:27.826736 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 982/1981] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:35:27.833657 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c03a06-fa25-4725-979b-db732bb7d851 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:27.846018 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c03a06-fa25-4725-979b-db732bb7d851 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:27.850974 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afc372-e8b6-4897-9eae-7edd15ee20bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:27.852123 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afc372-e8b6-4897-9eae-7edd15ee20bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:27.852432 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afc372-e8b6-4897-9eae-7edd15ee20bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:27.852678 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afc372-e8b6-4897-9eae-7edd15ee20bc admin admin] PATH_INFO: `/v3/projects/700cea58747346449902b9cd60948e5b/users/85234e3d7b0c4a4d9df4688c7143101b/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:27.871235 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59afc372-e8b6-4897-9eae-7edd15ee20bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=700cea58747346449902b9cd60948e5b, user_id=85234e3d7b0c4a4d9df4688c7143101b, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:27.874248 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59afc372-e8b6-4897-9eae-7edd15ee20bc admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:28.028198 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1000/1982] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:35:27 2022] PUT /identity/v3/projects/700cea58747346449902b9cd60948e5b/users/85234e3d7b0c4a4d9df4688c7143101b/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:28.040414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59748fae-8341-4f73-b3ff-d0fabb751662 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:28.043070 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59748fae-8341-4f73-b3ff-d0fabb751662 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:28.043070 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59748fae-8341-4f73-b3ff-d0fabb751662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:28.077587 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-59748fae-8341-4f73-b3ff-d0fabb751662 None None] MFA Rules not processed for user `85234e3d7b0c4a4d9df4688c7143101b`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:28.079046 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-59748fae-8341-4f73-b3ff-d0fabb751662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:28.175143 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 983/1983] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:35:28.189638 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afc372-e8b6-4897-9eae-7edd15ee20bc admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:28.326842 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59afc372-e8b6-4897-9eae-7edd15ee20bc admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:28.331900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246faa3-9fe9-4cc5-9442-8ff27c22ed7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:28.332681 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246faa3-9fe9-4cc5-9442-8ff27c22ed7b admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:28.332882 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246faa3-9fe9-4cc5-9442-8ff27c22ed7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:28.333163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246faa3-9fe9-4cc5-9442-8ff27c22ed7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:28.334415 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246faa3-9fe9-4cc5-9442-8ff27c22ed7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:28.336353 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246faa3-9fe9-4cc5-9442-8ff27c22ed7b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:28.481371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1001/1984] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:35:28 2022] POST /identity/v3/projects => generated 377 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:35:28.482457 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59748fae-8341-4f73-b3ff-d0fabb751662 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:28.494574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59748fae-8341-4f73-b3ff-d0fabb751662 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:28.581765 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db975b-b54a-42ab-bda5-364cd200cb05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:28.582593 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db975b-b54a-42ab-bda5-364cd200cb05 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:28.582943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db975b-b54a-42ab-bda5-364cd200cb05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:28.583298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db975b-b54a-42ab-bda5-364cd200cb05 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:28.584254 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db975b-b54a-42ab-bda5-364cd200cb05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:28.587049 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db975b-b54a-42ab-bda5-364cd200cb05 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:28.710022 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 984/1985] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:35:28 2022] POST /identity/v3/users => generated 295 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:35:28.714033 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246faa3-9fe9-4cc5-9442-8ff27c22ed7b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:28.728057 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246faa3-9fe9-4cc5-9442-8ff27c22ed7b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:28.732930 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bae407-4ac4-4959-8d12-0867823c2b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:28.733893 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bae407-4ac4-4959-8d12-0867823c2b8c admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:28.734184 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bae407-4ac4-4959-8d12-0867823c2b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:28.734462 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bae407-4ac4-4959-8d12-0867823c2b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:28.735196 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bae407-4ac4-4959-8d12-0867823c2b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:28.736961 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bae407-4ac4-4959-8d12-0867823c2b8c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:28.755981 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1002/1986] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:35:28.757355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db975b-b54a-42ab-bda5-364cd200cb05 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:28.770159 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db975b-b54a-42ab-bda5-364cd200cb05 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:28.782684 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4626c-3385-42f9-9e05-9b661ff75b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:28.782684 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4626c-3385-42f9-9e05-9b661ff75b97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:28.782684 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4626c-3385-42f9-9e05-9b661ff75b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:28.782684 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac4626c-3385-42f9-9e05-9b661ff75b97 admin admin] PATH_INFO: `/v3/projects/c6c3da678b484b079729c603fcbb6f6b/users/f96ede345afc4e2082eea8ae130990f6/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:28.809143 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac4626c-3385-42f9-9e05-9b661ff75b97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6c3da678b484b079729c603fcbb6f6b, user_id=f96ede345afc4e2082eea8ae130990f6, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:28.817156 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac4626c-3385-42f9-9e05-9b661ff75b97 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:28.934505 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 985/1987] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:35:28 2022] PUT /identity/v3/projects/c6c3da678b484b079729c603fcbb6f6b/users/f96ede345afc4e2082eea8ae130990f6/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:35:28.938405 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bae407-4ac4-4959-8d12-0867823c2b8c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:28.950687 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bae407-4ac4-4959-8d12-0867823c2b8c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:28.968898 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:29.014505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65c9ced-d459-4c37-9163-cb8f4d9cdafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:29.015184 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c9ced-d459-4c37-9163-cb8f4d9cdafe admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:29.015502 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c9ced-d459-4c37-9163-cb8f4d9cdafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:29.015827 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c9ced-d459-4c37-9163-cb8f4d9cdafe admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:29.016692 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65c9ced-d459-4c37-9163-cb8f4d9cdafe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:29.018317 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65c9ced-d459-4c37-9163-cb8f4d9cdafe admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:29.029871 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1003/1988] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:35:29.034335 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4626c-3385-42f9-9e05-9b661ff75b97 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:29.045386 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac4626c-3385-42f9-9e05-9b661ff75b97 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:29.050058 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4b7b9-75fe-4e32-85ba-0c17fd8d3cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:29.051018 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4b7b9-75fe-4e32-85ba-0c17fd8d3cc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:29.051300 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4b7b9-75fe-4e32-85ba-0c17fd8d3cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:29.051718 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4b7b9-75fe-4e32-85ba-0c17fd8d3cc4 admin admin] PATH_INFO: `/v3/projects/c6c3da678b484b079729c603fcbb6f6b/users/f96ede345afc4e2082eea8ae130990f6/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:29.053819 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b4b7b9-75fe-4e32-85ba-0c17fd8d3cc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6c3da678b484b079729c603fcbb6f6b, user_id=f96ede345afc4e2082eea8ae130990f6, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:29.056785 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b4b7b9-75fe-4e32-85ba-0c17fd8d3cc4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:29.063935 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:29.080966 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 986/1989] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:35:29 2022] PUT /identity/v3/projects/c6c3da678b484b079729c603fcbb6f6b/users/f96ede345afc4e2082eea8ae130990f6/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:29.093436 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd1fcc-7ef3-4f21-8404-46f10c34fa93 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:29.093980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd1fcc-7ef3-4f21-8404-46f10c34fa93 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:29.094498 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd1fcc-7ef3-4f21-8404-46f10c34fa93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:29.143754 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a5cd1fcc-7ef3-4f21-8404-46f10c34fa93 None None] MFA Rules not processed for user `f96ede345afc4e2082eea8ae130990f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:29.145896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a5cd1fcc-7ef3-4f21-8404-46f10c34fa93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:29.266453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1004/1990] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:29.279853 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4b7b9-75fe-4e32-85ba-0c17fd8d3cc4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:29.285683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e0b4b7b9-75fe-4e32-85ba-0c17fd8d3cc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:29.349960 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4b7b9-75fe-4e32-85ba-0c17fd8d3cc4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:29.354593 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a8c5e-ce2b-4e02-90ab-dc696745b8c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:29.355256 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a8c5e-ce2b-4e02-90ab-dc696745b8c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:29.355575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a8c5e-ce2b-4e02-90ab-dc696745b8c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:29.355845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a8c5e-ce2b-4e02-90ab-dc696745b8c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:29.420038 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a8c5e-ce2b-4e02-90ab-dc696745b8c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:29.422199 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a8c5e-ce2b-4e02-90ab-dc696745b8c2 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:29.439992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 987/1991] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:35:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:35:32.951748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd1fcc-7ef3-4f21-8404-46f10c34fa93 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:32.953619 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a5cd1fcc-7ef3-4f21-8404-46f10c34fa93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:33.035234 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd1fcc-7ef3-4f21-8404-46f10c34fa93 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:33.040673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5ce02-f1aa-470e-b6c3-ec7908e60c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:33.041629 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5ce02-f1aa-470e-b6c3-ec7908e60c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:33.041896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5ce02-f1aa-470e-b6c3-ec7908e60c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:33.042195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5ce02-f1aa-470e-b6c3-ec7908e60c4b admin admin] PATH_INFO: `/v3/users/939467dd5f9f420ebd37665d6afae392` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:33.043609 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad5ce02-f1aa-470e-b6c3-ec7908e60c4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=939467dd5f9f420ebd37665d6afae392)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:33.046018 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad5ce02-f1aa-470e-b6c3-ec7908e60c4b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:33.326537 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:33.333595 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-dad5ce02-f1aa-470e-b6c3-ec7908e60c4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '939467dd5f9f420ebd37665d6afae392'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:33.335144 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-dad5ce02-f1aa-470e-b6c3-ec7908e60c4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '939467dd5f9f420ebd37665d6afae392'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:33.341754 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-dad5ce02-f1aa-470e-b6c3-ec7908e60c4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '939467dd5f9f420ebd37665d6afae392'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:33.449083 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-dad5ce02-f1aa-470e-b6c3-ec7908e60c4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '939467dd5f9f420ebd37665d6afae392'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:33.449688 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-dad5ce02-f1aa-470e-b6c3-ec7908e60c4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '939467dd5f9f420ebd37665d6afae392'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:33.469889 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1005/1992] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:35:32 2022] DELETE /identity/v3/users/939467dd5f9f420ebd37665d6afae392 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:35:33.474658 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5be8f-caf2-407e-8a45-b9088577b01e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:33.475166 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5be8f-caf2-407e-8a45-b9088577b01e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:33.475266 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a5be8f-caf2-407e-8a45-b9088577b01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:33.494582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-55a5be8f-caf2-407e-8a45-b9088577b01e None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:33.495865 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-55a5be8f-caf2-407e-8a45-b9088577b01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:33.560622 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 988/1993] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:35:33.570579 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5ce02-f1aa-470e-b6c3-ec7908e60c4b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:33.572844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-dad5ce02-f1aa-470e-b6c3-ec7908e60c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:33.644338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5ce02-f1aa-470e-b6c3-ec7908e60c4b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:33.648033 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e666c261-1ade-443d-aece-0eb4ffad4109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:33.648725 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666c261-1ade-443d-aece-0eb4ffad4109 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:33.649009 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666c261-1ade-443d-aece-0eb4ffad4109 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:33.649295 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e666c261-1ade-443d-aece-0eb4ffad4109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:33.659951 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e666c261-1ade-443d-aece-0eb4ffad4109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:33.661417 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e666c261-1ade-443d-aece-0eb4ffad4109 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:33.672385 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1006/1994] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:35:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:35:34.126960 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a5be8f-caf2-407e-8a45-b9088577b01e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:34.130279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-55a5be8f-caf2-407e-8a45-b9088577b01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:34.152467 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a5be8f-caf2-407e-8a45-b9088577b01e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:34.163209 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92428657-b172-4f34-930d-d66cc7c19d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:34.163209 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92428657-b172-4f34-930d-d66cc7c19d9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:34.163209 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92428657-b172-4f34-930d-d66cc7c19d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:34.163209 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92428657-b172-4f34-930d-d66cc7c19d9f admin admin] PATH_INFO: `/v3/projects/ecc2efc61aad4698aaf73cb01b98bf08` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:34.163779 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92428657-b172-4f34-930d-d66cc7c19d9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecc2efc61aad4698aaf73cb01b98bf08)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:34.163779 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92428657-b172-4f34-930d-d66cc7c19d9f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:34.288652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:INSERT=1,keystone:SELECT=96,keystone:DELETE=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:34.295995 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-92428657-b172-4f34-930d-d66cc7c19d9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecc2efc61aad4698aaf73cb01b98bf08', 'request_id': 'req-92428657-b172-4f34-930d-d66cc7c19d9f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:34.306470 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-92428657-b172-4f34-930d-d66cc7c19d9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecc2efc61aad4698aaf73cb01b98bf08', 'request_id': 'req-92428657-b172-4f34-930d-d66cc7c19d9f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:34.315138 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-92428657-b172-4f34-930d-d66cc7c19d9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecc2efc61aad4698aaf73cb01b98bf08', 'request_id': 'req-92428657-b172-4f34-930d-d66cc7c19d9f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:34.383589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-92428657-b172-4f34-930d-d66cc7c19d9f admin admin] The token cache is being invalidate because project ecc2efc61aad4698aaf73cb01b98bf08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:35:34.384448 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-92428657-b172-4f34-930d-d66cc7c19d9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:34.385856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-92428657-b172-4f34-930d-d66cc7c19d9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:34.387547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 989/1995] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:35:34 2022] DELETE /identity/v3/projects/ecc2efc61aad4698aaf73cb01b98bf08 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:34.392705 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc3d8b-3f7e-4f5c-a3d5-ee2772ba1f1a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:34.393210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc3d8b-3f7e-4f5c-a3d5-ee2772ba1f1a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:34.393322 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abc3d8b-3f7e-4f5c-a3d5-ee2772ba1f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:34.413365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5abc3d8b-3f7e-4f5c-a3d5-ee2772ba1f1a None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:34.414749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5abc3d8b-3f7e-4f5c-a3d5-ee2772ba1f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:34.485210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1007/1996] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:35:34.493180 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d574992-6ecf-425b-aa3b-b2a0710cb6b1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:34.493728 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d574992-6ecf-425b-aa3b-b2a0710cb6b1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:34.493855 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d574992-6ecf-425b-aa3b-b2a0710cb6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:34.514764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5d574992-6ecf-425b-aa3b-b2a0710cb6b1 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:34.516101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5d574992-6ecf-425b-aa3b-b2a0710cb6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:34.524278 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 990/1997] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:34.530225 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc3d8b-3f7e-4f5c-a3d5-ee2772ba1f1a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:34.540803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abc3d8b-3f7e-4f5c-a3d5-ee2772ba1f1a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:34.550141 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad532cf1-84f8-4a7b-b584-ad404dd46f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:34.550141 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad532cf1-84f8-4a7b-b584-ad404dd46f14 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:34.550141 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad532cf1-84f8-4a7b-b584-ad404dd46f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:34.550141 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad532cf1-84f8-4a7b-b584-ad404dd46f14 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:34.550529 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad532cf1-84f8-4a7b-b584-ad404dd46f14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:35:34.550529 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad532cf1-84f8-4a7b-b584-ad404dd46f14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:34.550529 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad532cf1-84f8-4a7b-b584-ad404dd46f14 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:34.561811 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1008/1998] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:35:34.566248 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d574992-6ecf-425b-aa3b-b2a0710cb6b1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:34.579029 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d574992-6ecf-425b-aa3b-b2a0710cb6b1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:34.581680 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3027b4-d83c-4226-b354-48ef83e722fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:34.582427 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3027b4-d83c-4226-b354-48ef83e722fc admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:34.582710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3027b4-d83c-4226-b354-48ef83e722fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:34.582988 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3027b4-d83c-4226-b354-48ef83e722fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:34.583713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3027b4-d83c-4226-b354-48ef83e722fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:34.585625 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3027b4-d83c-4226-b354-48ef83e722fc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:34.698118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 991/1999] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:35:34 2022] POST /identity/v3/projects => generated 369 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:35:34.703600 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad532cf1-84f8-4a7b-b584-ad404dd46f14 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:34.715885 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad532cf1-84f8-4a7b-b584-ad404dd46f14 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:34.767186 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:34.779456 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56d5fd-601d-43a9-8452-de389333eef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:34.780186 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56d5fd-601d-43a9-8452-de389333eef1 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:34.780476 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56d5fd-601d-43a9-8452-de389333eef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:34.780759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56d5fd-601d-43a9-8452-de389333eef1 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:34.781466 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56d5fd-601d-43a9-8452-de389333eef1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:34.783717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56d5fd-601d-43a9-8452-de389333eef1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:34.937374 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1009/2000] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:35:34 2022] POST /identity/v3/users => generated 285 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:35:34.942232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3027b4-d83c-4226-b354-48ef83e722fc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:34.954223 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3027b4-d83c-4226-b354-48ef83e722fc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:34.958883 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eff21e-5306-44a7-9f2d-c59ca1b51799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:34.959601 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eff21e-5306-44a7-9f2d-c59ca1b51799 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:34.959886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eff21e-5306-44a7-9f2d-c59ca1b51799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:34.960233 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eff21e-5306-44a7-9f2d-c59ca1b51799 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:34.960959 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eff21e-5306-44a7-9f2d-c59ca1b51799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:34.962613 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8eff21e-5306-44a7-9f2d-c59ca1b51799 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:34.973728 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 992/2001] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:35:34.978622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56d5fd-601d-43a9-8452-de389333eef1 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:34.998534 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56d5fd-601d-43a9-8452-de389333eef1 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:35.008301 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c44951-bd23-4297-8863-70765a976a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:35.010019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c44951-bd23-4297-8863-70765a976a5f admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:35.010407 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c44951-bd23-4297-8863-70765a976a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:35.011417 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c44951-bd23-4297-8863-70765a976a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:35.012300 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c44951-bd23-4297-8863-70765a976a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:35.015776 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c44951-bd23-4297-8863-70765a976a5f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:35.034330 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1010/2002] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:35:35.044964 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eff21e-5306-44a7-9f2d-c59ca1b51799 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:35.059686 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8eff21e-5306-44a7-9f2d-c59ca1b51799 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:35.071133 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784b598-6bd3-4bee-9b5f-3cf6bf7267a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:35.072842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784b598-6bd3-4bee-9b5f-3cf6bf7267a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:35.073408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784b598-6bd3-4bee-9b5f-3cf6bf7267a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:35.073915 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784b598-6bd3-4bee-9b5f-3cf6bf7267a5 admin admin] PATH_INFO: `/v3/projects/7182357d113a41a6a077dd543e48c767/users/8e5805e139e141109cee1cacba204c55/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:35.097082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784b598-6bd3-4bee-9b5f-3cf6bf7267a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7182357d113a41a6a077dd543e48c767, user_id=8e5805e139e141109cee1cacba204c55, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:35.099783 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784b598-6bd3-4bee-9b5f-3cf6bf7267a5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:35.201953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 993/2003] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:35:35 2022] PUT /identity/v3/projects/7182357d113a41a6a077dd543e48c767/users/8e5805e139e141109cee1cacba204c55/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:35.214527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6359a6b-2314-4ec9-a95d-a803c98ec46d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:35.214839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6359a6b-2314-4ec9-a95d-a803c98ec46d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:35.219378 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6359a6b-2314-4ec9-a95d-a803c98ec46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:35.247203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a6359a6b-2314-4ec9-a95d-a803c98ec46d None None] MFA Rules not processed for user `8e5805e139e141109cee1cacba204c55`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:35.248612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a6359a6b-2314-4ec9-a95d-a803c98ec46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:35.316035 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1011/2004] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:35 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:35.323593 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784b598-6bd3-4bee-9b5f-3cf6bf7267a5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:35.391424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784b598-6bd3-4bee-9b5f-3cf6bf7267a5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:35.396485 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2365b-5bef-41ce-9262-f72b764864b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:35.397157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2365b-5bef-41ce-9262-f72b764864b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:35.397441 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2365b-5bef-41ce-9262-f72b764864b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:35.397722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b2365b-5bef-41ce-9262-f72b764864b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:35.398433 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b2365b-5bef-41ce-9262-f72b764864b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:35.400875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b2365b-5bef-41ce-9262-f72b764864b7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:35.577118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 994/2005] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:35:35 2022] POST /identity/v3/projects => generated 369 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:35:35.582806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6359a6b-2314-4ec9-a95d-a803c98ec46d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:35.594637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6359a6b-2314-4ec9-a95d-a803c98ec46d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:35.653168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91848052-d833-4d8d-a5f7-d7867fe5e96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:35.653879 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91848052-d833-4d8d-a5f7-d7867fe5e96d admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:35.654166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91848052-d833-4d8d-a5f7-d7867fe5e96d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:35.654447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91848052-d833-4d8d-a5f7-d7867fe5e96d admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:35.655153 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91848052-d833-4d8d-a5f7-d7867fe5e96d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:35.657464 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91848052-d833-4d8d-a5f7-d7867fe5e96d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:35.753161 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1012/2006] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:35:35 2022] POST /identity/v3/users => generated 291 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:35:35.758602 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2365b-5bef-41ce-9262-f72b764864b7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:35.770084 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b2365b-5bef-41ce-9262-f72b764864b7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:35.775579 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb783411-a1cb-4136-b013-07834eabc121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:35.779283 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb783411-a1cb-4136-b013-07834eabc121 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:35.779283 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb783411-a1cb-4136-b013-07834eabc121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:35.779283 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb783411-a1cb-4136-b013-07834eabc121 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:35.779283 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb783411-a1cb-4136-b013-07834eabc121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:35.779283 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb783411-a1cb-4136-b013-07834eabc121 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:35.790469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 995/2007] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:35:35.794820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91848052-d833-4d8d-a5f7-d7867fe5e96d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:35.805835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91848052-d833-4d8d-a5f7-d7867fe5e96d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:35.810714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13514731-4fbb-4915-abd7-9d580576941c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:35.811639 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13514731-4fbb-4915-abd7-9d580576941c admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:35.811922 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13514731-4fbb-4915-abd7-9d580576941c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:35.812196 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13514731-4fbb-4915-abd7-9d580576941c admin admin] PATH_INFO: `/v3/projects/f227c75eb8ec4caab496ad10d6bf3599/users/746c742d8fa4482fa7c7a9e9d5eb9766/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:35.828758 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13514731-4fbb-4915-abd7-9d580576941c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f227c75eb8ec4caab496ad10d6bf3599, user_id=746c742d8fa4482fa7c7a9e9d5eb9766, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:35.831558 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13514731-4fbb-4915-abd7-9d580576941c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:35.930348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1013/2008] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:35:35 2022] PUT /identity/v3/projects/f227c75eb8ec4caab496ad10d6bf3599/users/746c742d8fa4482fa7c7a9e9d5eb9766/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:35.934562 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb783411-a1cb-4136-b013-07834eabc121 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:35.945394 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb783411-a1cb-4136-b013-07834eabc121 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:35.989604 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:36.006430 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c0240-f825-4842-80d2-16b8a185894d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:36.007042 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c0240-f825-4842-80d2-16b8a185894d admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:36.007328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c0240-f825-4842-80d2-16b8a185894d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:36.008628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c0240-f825-4842-80d2-16b8a185894d admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:36.008628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c0240-f825-4842-80d2-16b8a185894d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:36.010734 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c0240-f825-4842-80d2-16b8a185894d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:36.021071 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 996/2009] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:35:36.026243 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13514731-4fbb-4915-abd7-9d580576941c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:36.037414 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13514731-4fbb-4915-abd7-9d580576941c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:36.042354 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038914d4-88f2-45bd-b02f-c01d69477597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:36.043243 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038914d4-88f2-45bd-b02f-c01d69477597 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:36.043570 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038914d4-88f2-45bd-b02f-c01d69477597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:36.043853 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038914d4-88f2-45bd-b02f-c01d69477597 admin admin] PATH_INFO: `/v3/projects/f227c75eb8ec4caab496ad10d6bf3599/users/746c742d8fa4482fa7c7a9e9d5eb9766/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:36.045916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038914d4-88f2-45bd-b02f-c01d69477597 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f227c75eb8ec4caab496ad10d6bf3599, user_id=746c742d8fa4482fa7c7a9e9d5eb9766, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:36.048756 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038914d4-88f2-45bd-b02f-c01d69477597 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:36.060271 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1014/2010] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:35:36 2022] PUT /identity/v3/projects/f227c75eb8ec4caab496ad10d6bf3599/users/746c742d8fa4482fa7c7a9e9d5eb9766/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:36.070680 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15609c-a380-44cf-9285-48a075611313 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:36.070995 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15609c-a380-44cf-9285-48a075611313 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:36.071228 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15609c-a380-44cf-9285-48a075611313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:36.100764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2f15609c-a380-44cf-9285-48a075611313 None None] MFA Rules not processed for user `746c742d8fa4482fa7c7a9e9d5eb9766`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:36.102347 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2f15609c-a380-44cf-9285-48a075611313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:36.183769 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 997/2011] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:36 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:36.188635 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038914d4-88f2-45bd-b02f-c01d69477597 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:36.190523 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-038914d4-88f2-45bd-b02f-c01d69477597 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:36.195126 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15609c-a380-44cf-9285-48a075611313 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:36.196481 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2f15609c-a380-44cf-9285-48a075611313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:36.205512 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:36.254787 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038914d4-88f2-45bd-b02f-c01d69477597 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:36.258504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e937804-8eed-4aed-9350-7508a4c9c0cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:36.259143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e937804-8eed-4aed-9350-7508a4c9c0cb service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:36.260029 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e937804-8eed-4aed-9350-7508a4c9c0cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:36.260304 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e937804-8eed-4aed-9350-7508a4c9c0cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:36.261184 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15609c-a380-44cf-9285-48a075611313 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:36.262140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8e937804-8eed-4aed-9350-7508a4c9c0cb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:36.264807 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61aea8-dcf8-4cb6-81f4-f55872474ac5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:36.265445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61aea8-dcf8-4cb6-81f4-f55872474ac5 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:36.265728 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61aea8-dcf8-4cb6-81f4-f55872474ac5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:36.265992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61aea8-dcf8-4cb6-81f4-f55872474ac5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:36.276955 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61aea8-dcf8-4cb6-81f4-f55872474ac5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:36.278246 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61aea8-dcf8-4cb6-81f4-f55872474ac5 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:36.289386 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 998/2012] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:35:36 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:35:36.333573 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e937804-8eed-4aed-9350-7508a4c9c0cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:36.335127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e937804-8eed-4aed-9350-7508a4c9c0cb service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:36.345203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1015/2013] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:35:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:35:36.421152 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61aea8-dcf8-4cb6-81f4-f55872474ac5 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:36.429976 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61aea8-dcf8-4cb6-81f4-f55872474ac5 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:36.434846 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec603632-c398-49ff-8a89-edeb0e8fe06f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:36.437977 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec603632-c398-49ff-8a89-edeb0e8fe06f service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:36.437977 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec603632-c398-49ff-8a89-edeb0e8fe06f service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:36.437977 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec603632-c398-49ff-8a89-edeb0e8fe06f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:36.496811 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec603632-c398-49ff-8a89-edeb0e8fe06f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:36.498264 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec603632-c398-49ff-8a89-edeb0e8fe06f service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:36.507735 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 999/2014] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:35:36 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:35:42.182068 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e937804-8eed-4aed-9350-7508a4c9c0cb service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:42.321270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e937804-8eed-4aed-9350-7508a4c9c0cb service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:42.327181 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c754c-666f-4926-a106-da59dcb3e205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:42.328123 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c754c-666f-4926-a106-da59dcb3e205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:42.328402 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c754c-666f-4926-a106-da59dcb3e205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:42.328677 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254c754c-666f-4926-a106-da59dcb3e205 admin admin] PATH_INFO: `/v3/users/8e5805e139e141109cee1cacba204c55` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:42.330010 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c754c-666f-4926-a106-da59dcb3e205 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e5805e139e141109cee1cacba204c55)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:42.332146 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254c754c-666f-4926-a106-da59dcb3e205 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:42.364174 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:42.653512 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-254c754c-666f-4926-a106-da59dcb3e205 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e5805e139e141109cee1cacba204c55'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:42.654085 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-254c754c-666f-4926-a106-da59dcb3e205 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e5805e139e141109cee1cacba204c55'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:42.662920 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-254c754c-666f-4926-a106-da59dcb3e205 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e5805e139e141109cee1cacba204c55'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:42.768683 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-254c754c-666f-4926-a106-da59dcb3e205 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e5805e139e141109cee1cacba204c55'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:42.769665 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-254c754c-666f-4926-a106-da59dcb3e205 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e5805e139e141109cee1cacba204c55'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:42.792847 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1016/2015] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:35:42 2022] DELETE /identity/v3/users/8e5805e139e141109cee1cacba204c55 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:42.802102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6d6e1-9b6b-4fe8-88a4-897de7f7a357 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:42.802638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6d6e1-9b6b-4fe8-88a4-897de7f7a357 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:42.802784 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6d6e1-9b6b-4fe8-88a4-897de7f7a357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:42.808454 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:42.860604 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e1c6d6e1-9b6b-4fe8-88a4-897de7f7a357 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:42.861734 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e1c6d6e1-9b6b-4fe8-88a4-897de7f7a357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:42.928334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1000/2016] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:42.942997 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c754c-666f-4926-a106-da59dcb3e205 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:42.944636 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-254c754c-666f-4926-a106-da59dcb3e205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:43.060887 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254c754c-666f-4926-a106-da59dcb3e205 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:43.065873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30687784-32cf-465e-99a0-56ffc7569b98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:43.066673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30687784-32cf-465e-99a0-56ffc7569b98 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:43.066959 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30687784-32cf-465e-99a0-56ffc7569b98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:43.067237 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30687784-32cf-465e-99a0-56ffc7569b98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:43.078136 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30687784-32cf-465e-99a0-56ffc7569b98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:43.079647 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30687784-32cf-465e-99a0-56ffc7569b98 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:43.090882 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1017/2017] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:35:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:35:43.667230 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c6d6e1-9b6b-4fe8-88a4-897de7f7a357 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:43.669666 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e1c6d6e1-9b6b-4fe8-88a4-897de7f7a357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:43.683968 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c6d6e1-9b6b-4fe8-88a4-897de7f7a357 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:43.693950 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0751b-7c07-4057-a3f4-8e6221c47469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:43.695882 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0751b-7c07-4057-a3f4-8e6221c47469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:43.696170 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0751b-7c07-4057-a3f4-8e6221c47469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:43.696439 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0751b-7c07-4057-a3f4-8e6221c47469 admin admin] PATH_INFO: `/v3/projects/7182357d113a41a6a077dd543e48c767` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:43.697763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0751b-7c07-4057-a3f4-8e6221c47469 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7182357d113a41a6a077dd543e48c767)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:43.701088 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f0751b-7c07-4057-a3f4-8e6221c47469 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:43.855829 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e6f0751b-7c07-4057-a3f4-8e6221c47469 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7182357d113a41a6a077dd543e48c767', 'request_id': 'req-e6f0751b-7c07-4057-a3f4-8e6221c47469'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:43.863425 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e6f0751b-7c07-4057-a3f4-8e6221c47469 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7182357d113a41a6a077dd543e48c767', 'request_id': 'req-e6f0751b-7c07-4057-a3f4-8e6221c47469'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:43.873248 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e6f0751b-7c07-4057-a3f4-8e6221c47469 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7182357d113a41a6a077dd543e48c767', 'request_id': 'req-e6f0751b-7c07-4057-a3f4-8e6221c47469'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:44.008294 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e6f0751b-7c07-4057-a3f4-8e6221c47469 admin admin] The token cache is being invalidate because project 7182357d113a41a6a077dd543e48c767 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:35:44.008294 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e6f0751b-7c07-4057-a3f4-8e6221c47469 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:44.008294 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e6f0751b-7c07-4057-a3f4-8e6221c47469 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:44.017626 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1001/2018] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:35:43 2022] DELETE /identity/v3/projects/7182357d113a41a6a077dd543e48c767 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:44.020526 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30687784-32cf-465e-99a0-56ffc7569b98 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:44.035247 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30687784-32cf-465e-99a0-56ffc7569b98 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:44.104669 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da151b-0f74-4d47-92ab-7407101d0c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:44.106505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da151b-0f74-4d47-92ab-7407101d0c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:44.106718 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da151b-0f74-4d47-92ab-7407101d0c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:44.107005 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da151b-0f74-4d47-92ab-7407101d0c9d admin admin] PATH_INFO: `/v3/users/746c742d8fa4482fa7c7a9e9d5eb9766` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:44.108477 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da151b-0f74-4d47-92ab-7407101d0c9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=746c742d8fa4482fa7c7a9e9d5eb9766)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:44.110455 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da151b-0f74-4d47-92ab-7407101d0c9d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:44.116494 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:44.400491 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-55da151b-0f74-4d47-92ab-7407101d0c9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '746c742d8fa4482fa7c7a9e9d5eb9766'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:44.401171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-55da151b-0f74-4d47-92ab-7407101d0c9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '746c742d8fa4482fa7c7a9e9d5eb9766'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:44.410015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-55da151b-0f74-4d47-92ab-7407101d0c9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '746c742d8fa4482fa7c7a9e9d5eb9766'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:44.494284 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-55da151b-0f74-4d47-92ab-7407101d0c9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '746c742d8fa4482fa7c7a9e9d5eb9766'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:44.495082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-55da151b-0f74-4d47-92ab-7407101d0c9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '746c742d8fa4482fa7c7a9e9d5eb9766'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:44.517665 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1018/2019] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:35:44 2022] DELETE /identity/v3/users/746c742d8fa4482fa7c7a9e9d5eb9766 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:45.076844 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0751b-7c07-4057-a3f4-8e6221c47469 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:45.079358 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e6f0751b-7c07-4057-a3f4-8e6221c47469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:45.095569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0751b-7c07-4057-a3f4-8e6221c47469 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:45.160040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2184a8-5c88-4b19-a999-3dc5ca05b300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:45.160922 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2184a8-5c88-4b19-a999-3dc5ca05b300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:45.161208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2184a8-5c88-4b19-a999-3dc5ca05b300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:45.161485 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2184a8-5c88-4b19-a999-3dc5ca05b300 admin admin] PATH_INFO: `/v3/projects/f227c75eb8ec4caab496ad10d6bf3599` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:45.162728 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2184a8-5c88-4b19-a999-3dc5ca05b300 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f227c75eb8ec4caab496ad10d6bf3599)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:45.164757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2184a8-5c88-4b19-a999-3dc5ca05b300 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:45.167858 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:45.420047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9f2184a8-5c88-4b19-a999-3dc5ca05b300 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f227c75eb8ec4caab496ad10d6bf3599', 'request_id': 'req-9f2184a8-5c88-4b19-a999-3dc5ca05b300'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:45.429036 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9f2184a8-5c88-4b19-a999-3dc5ca05b300 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f227c75eb8ec4caab496ad10d6bf3599', 'request_id': 'req-9f2184a8-5c88-4b19-a999-3dc5ca05b300'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:45.443760 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9f2184a8-5c88-4b19-a999-3dc5ca05b300 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f227c75eb8ec4caab496ad10d6bf3599', 'request_id': 'req-9f2184a8-5c88-4b19-a999-3dc5ca05b300'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:45.559678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9f2184a8-5c88-4b19-a999-3dc5ca05b300 admin admin] The token cache is being invalidate because project f227c75eb8ec4caab496ad10d6bf3599 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:35:45.559954 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9f2184a8-5c88-4b19-a999-3dc5ca05b300 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:45.560607 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-9f2184a8-5c88-4b19-a999-3dc5ca05b300 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:45.562226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1002/2020] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:35:45 2022] DELETE /identity/v3/projects/f227c75eb8ec4caab496ad10d6bf3599 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:35:45.567287 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed6e8a-b44c-4921-93ea-4f9a6c8e3ea7 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:45.567791 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed6e8a-b44c-4921-93ea-4f9a6c8e3ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:45.567885 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aed6e8a-b44c-4921-93ea-4f9a6c8e3ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:45.605909 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-4aed6e8a-b44c-4921-93ea-4f9a6c8e3ea7 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:45.607215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4aed6e8a-b44c-4921-93ea-4f9a6c8e3ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:45.671298 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1019/2021] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:45.681389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014aebb-0ab4-4b09-b163-047366e5f542 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:45.681889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014aebb-0ab4-4b09-b163-047366e5f542 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:45.682392 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1014aebb-0ab4-4b09-b163-047366e5f542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:45.715684 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1014aebb-0ab4-4b09-b163-047366e5f542 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:45.717070 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1014aebb-0ab4-4b09-b163-047366e5f542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:45.725667 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1003/2022] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:45.730086 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed6e8a-b44c-4921-93ea-4f9a6c8e3ea7 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:45.742624 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aed6e8a-b44c-4921-93ea-4f9a6c8e3ea7 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:45.748099 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5beceff-4072-403a-803b-061ca4a8d57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:45.748812 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5beceff-4072-403a-803b-061ca4a8d57f admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:45.749095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5beceff-4072-403a-803b-061ca4a8d57f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:45.749371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5beceff-4072-403a-803b-061ca4a8d57f admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:45.750144 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5beceff-4072-403a-803b-061ca4a8d57f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:35:45.750482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5beceff-4072-403a-803b-061ca4a8d57f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:45.752232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5beceff-4072-403a-803b-061ca4a8d57f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:45.768923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1020/2023] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:35:45.770284 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014aebb-0ab4-4b09-b163-047366e5f542 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:45.780153 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1014aebb-0ab4-4b09-b163-047366e5f542 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:45.790757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3297e50-06ed-45fe-bf8c-8f42d344892d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:45.790757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3297e50-06ed-45fe-bf8c-8f42d344892d admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:45.790757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3297e50-06ed-45fe-bf8c-8f42d344892d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:45.790757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3297e50-06ed-45fe-bf8c-8f42d344892d admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:45.791171 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3297e50-06ed-45fe-bf8c-8f42d344892d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:45.791171 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3297e50-06ed-45fe-bf8c-8f42d344892d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:45.964672 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1004/2024] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:35:45 2022] POST /identity/v3/projects => generated 399 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:35:45.969944 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5beceff-4072-403a-803b-061ca4a8d57f admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:45.983702 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5beceff-4072-403a-803b-061ca4a8d57f admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:46.095462 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:46.108185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd6eaf-04ce-46db-a31a-e4fa56283961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:46.109368 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd6eaf-04ce-46db-a31a-e4fa56283961 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:46.109875 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd6eaf-04ce-46db-a31a-e4fa56283961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:46.110384 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd6eaf-04ce-46db-a31a-e4fa56283961 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:46.111800 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd6eaf-04ce-46db-a31a-e4fa56283961 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:46.115692 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd6eaf-04ce-46db-a31a-e4fa56283961 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:46.257158 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1021/2025] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:35:45 2022] POST /identity/v3/users => generated 300 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:35:46.258787 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3297e50-06ed-45fe-bf8c-8f42d344892d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:46.271150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3297e50-06ed-45fe-bf8c-8f42d344892d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:46.276173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39db84-856f-41f5-9c2b-95e357325255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:46.276820 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39db84-856f-41f5-9c2b-95e357325255 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:46.277652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39db84-856f-41f5-9c2b-95e357325255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:46.277652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39db84-856f-41f5-9c2b-95e357325255 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:46.279027 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39db84-856f-41f5-9c2b-95e357325255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:46.280743 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39db84-856f-41f5-9c2b-95e357325255 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:46.292995 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1005/2026] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:35:46.298129 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd6eaf-04ce-46db-a31a-e4fa56283961 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:46.308380 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd6eaf-04ce-46db-a31a-e4fa56283961 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:46.313311 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4de62-d0ea-48de-aba2-2a99958d3a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:46.313982 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4de62-d0ea-48de-aba2-2a99958d3a1c admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:46.314268 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4de62-d0ea-48de-aba2-2a99958d3a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:46.314545 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4de62-d0ea-48de-aba2-2a99958d3a1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:46.315273 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df4de62-d0ea-48de-aba2-2a99958d3a1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:46.316932 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df4de62-d0ea-48de-aba2-2a99958d3a1c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:46.329999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1022/2027] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:35:46.334505 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39db84-856f-41f5-9c2b-95e357325255 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:46.346009 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39db84-856f-41f5-9c2b-95e357325255 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:46.351570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239528ba-8394-4212-b6de-7d448616eeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:46.352493 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239528ba-8394-4212-b6de-7d448616eeb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:46.357556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239528ba-8394-4212-b6de-7d448616eeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:46.357556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239528ba-8394-4212-b6de-7d448616eeb0 admin admin] PATH_INFO: `/v3/projects/eddad92f431b4341829cac52d5cf0ed7/users/1be16beb38474ae7b9f4b811552823c9/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:46.372932 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239528ba-8394-4212-b6de-7d448616eeb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eddad92f431b4341829cac52d5cf0ed7, user_id=1be16beb38474ae7b9f4b811552823c9, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:46.376010 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239528ba-8394-4212-b6de-7d448616eeb0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:46.475363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1006/2028] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:35:46 2022] PUT /identity/v3/projects/eddad92f431b4341829cac52d5cf0ed7/users/1be16beb38474ae7b9f4b811552823c9/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:35:46.486903 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82939c5a-5591-4560-a43c-3e8ca2d2197c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:46.487183 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82939c5a-5591-4560-a43c-3e8ca2d2197c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:46.487460 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82939c5a-5591-4560-a43c-3e8ca2d2197c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:46.517308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-82939c5a-5591-4560-a43c-3e8ca2d2197c None None] MFA Rules not processed for user `1be16beb38474ae7b9f4b811552823c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:46.518498 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-82939c5a-5591-4560-a43c-3e8ca2d2197c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:46.593817 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1023/2029] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:35:46.599030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239528ba-8394-4212-b6de-7d448616eeb0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:46.663555 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239528ba-8394-4212-b6de-7d448616eeb0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:46.668609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a9a645-58ce-4fa4-b13a-80b20056a7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:46.669285 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9a645-58ce-4fa4-b13a-80b20056a7b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:46.669588 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9a645-58ce-4fa4-b13a-80b20056a7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:46.669864 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a9a645-58ce-4fa4-b13a-80b20056a7b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:46.670625 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a9a645-58ce-4fa4-b13a-80b20056a7b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:46.672623 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a9a645-58ce-4fa4-b13a-80b20056a7b5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:46.787343 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1007/2030] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:35:46 2022] POST /identity/v3/projects => generated 399 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:35:46.791662 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82939c5a-5591-4560-a43c-3e8ca2d2197c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:46.804361 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82939c5a-5591-4560-a43c-3e8ca2d2197c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:46.868828 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b12277-4041-47af-b778-e3b1dbf83c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:46.869525 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b12277-4041-47af-b778-e3b1dbf83c29 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:46.869817 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b12277-4041-47af-b778-e3b1dbf83c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:46.870099 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b12277-4041-47af-b778-e3b1dbf83c29 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:46.870817 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b12277-4041-47af-b778-e3b1dbf83c29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:46.872876 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b12277-4041-47af-b778-e3b1dbf83c29 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:46.991985 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1024/2031] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:35:46 2022] POST /identity/v3/users => generated 306 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:35:46.996170 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a9a645-58ce-4fa4-b13a-80b20056a7b5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:47.007691 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a9a645-58ce-4fa4-b13a-80b20056a7b5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:47.012985 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7333dc-e36b-463d-b896-9416b3cf8590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:47.013825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7333dc-e36b-463d-b896-9416b3cf8590 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:47.013948 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7333dc-e36b-463d-b896-9416b3cf8590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:47.014204 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7333dc-e36b-463d-b896-9416b3cf8590 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:47.014924 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7333dc-e36b-463d-b896-9416b3cf8590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:47.016577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7333dc-e36b-463d-b896-9416b3cf8590 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:47.030060 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1008/2032] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:35:47.034562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b12277-4041-47af-b778-e3b1dbf83c29 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:47.048345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b12277-4041-47af-b778-e3b1dbf83c29 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:47.054251 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf33fd-0035-437c-9d4d-adfc1d87db77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:47.055164 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf33fd-0035-437c-9d4d-adfc1d87db77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:47.055641 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf33fd-0035-437c-9d4d-adfc1d87db77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:47.055886 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf33fd-0035-437c-9d4d-adfc1d87db77 admin admin] PATH_INFO: `/v3/projects/833a7542256341fd9d4cfac82cf1f9f1/users/a66f7549150d4f368239a7090e298a9a/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:47.076302 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf33fd-0035-437c-9d4d-adfc1d87db77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=833a7542256341fd9d4cfac82cf1f9f1, user_id=a66f7549150d4f368239a7090e298a9a, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:47.079705 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf33fd-0035-437c-9d4d-adfc1d87db77 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:47.169293 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1025/2033] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:35:47 2022] PUT /identity/v3/projects/833a7542256341fd9d4cfac82cf1f9f1/users/a66f7549150d4f368239a7090e298a9a/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:47.173342 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7333dc-e36b-463d-b896-9416b3cf8590 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:47.178599 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:47.184674 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7333dc-e36b-463d-b896-9416b3cf8590 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:47.249815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e23375-216e-4735-bac5-462b3fbeee3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:47.250498 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e23375-216e-4735-bac5-462b3fbeee3a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:47.250786 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e23375-216e-4735-bac5-462b3fbeee3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:47.251066 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e23375-216e-4735-bac5-462b3fbeee3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:47.251836 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e23375-216e-4735-bac5-462b3fbeee3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:47.253532 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e23375-216e-4735-bac5-462b3fbeee3a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:47.265070 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1009/2034] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:35:47.271734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf33fd-0035-437c-9d4d-adfc1d87db77 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:47.290082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf33fd-0035-437c-9d4d-adfc1d87db77 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:47.298655 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36857885-e2f5-4b1c-a71a-44e79c8f96df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:47.300177 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36857885-e2f5-4b1c-a71a-44e79c8f96df admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:47.300874 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36857885-e2f5-4b1c-a71a-44e79c8f96df admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:47.301338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36857885-e2f5-4b1c-a71a-44e79c8f96df admin admin] PATH_INFO: `/v3/projects/833a7542256341fd9d4cfac82cf1f9f1/users/a66f7549150d4f368239a7090e298a9a/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:47.304953 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36857885-e2f5-4b1c-a71a-44e79c8f96df admin admin] RBAC: Authorizing `identity:create_grant(project_id=833a7542256341fd9d4cfac82cf1f9f1, user_id=a66f7549150d4f368239a7090e298a9a, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:47.310803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36857885-e2f5-4b1c-a71a-44e79c8f96df admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:47.330072 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1026/2035] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:35:47 2022] PUT /identity/v3/projects/833a7542256341fd9d4cfac82cf1f9f1/users/a66f7549150d4f368239a7090e298a9a/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:47.346086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f537547-5836-4d83-9f4e-9df97c021f59 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:47.346469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f537547-5836-4d83-9f4e-9df97c021f59 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:47.347147 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f537547-5836-4d83-9f4e-9df97c021f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:47.379121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0f537547-5836-4d83-9f4e-9df97c021f59 None None] MFA Rules not processed for user `a66f7549150d4f368239a7090e298a9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:47.380503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0f537547-5836-4d83-9f4e-9df97c021f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:47.466210 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1010/2036] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:47 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:47.480584 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36857885-e2f5-4b1c-a71a-44e79c8f96df admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:47.483053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-36857885-e2f5-4b1c-a71a-44e79c8f96df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:47.510399 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:47.551839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36857885-e2f5-4b1c-a71a-44e79c8f96df admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:47.556307 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b5f26-2bd4-421e-9cef-15f3094367d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:47.556963 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b5f26-2bd4-421e-9cef-15f3094367d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:47.557240 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b5f26-2bd4-421e-9cef-15f3094367d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:47.557512 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b5f26-2bd4-421e-9cef-15f3094367d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:47.568778 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b5f26-2bd4-421e-9cef-15f3094367d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:47.570269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36b5f26-2bd4-421e-9cef-15f3094367d7 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:47.581086 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1027/2037] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:35:47 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:35:47.792367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f537547-5836-4d83-9f4e-9df97c021f59 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:47.803335 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f537547-5836-4d83-9f4e-9df97c021f59 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:47.806999 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eec6a9-7a7e-4c75-b4a0-535e3ef37f6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:47.807735 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eec6a9-7a7e-4c75-b4a0-535e3ef37f6e service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:47.816616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eec6a9-7a7e-4c75-b4a0-535e3ef37f6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:47.816616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eec6a9-7a7e-4c75-b4a0-535e3ef37f6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:47.884524 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eec6a9-7a7e-4c75-b4a0-535e3ef37f6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:47.885982 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eec6a9-7a7e-4c75-b4a0-535e3ef37f6e service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:47.895993 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1011/2038] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:35:47 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:35:50.854522 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b5f26-2bd4-421e-9cef-15f3094367d7 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:50.856066 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d36b5f26-2bd4-421e-9cef-15f3094367d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:50.928319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b5f26-2bd4-421e-9cef-15f3094367d7 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:50.933990 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813841b-45be-4c3f-b7b8-e084158ba1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:50.935398 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813841b-45be-4c3f-b7b8-e084158ba1bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:50.936619 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813841b-45be-4c3f-b7b8-e084158ba1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:50.936790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813841b-45be-4c3f-b7b8-e084158ba1bf admin admin] PATH_INFO: `/v3/users/0973888f233c43e3bf2af18ebd01a932` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:50.937974 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8813841b-45be-4c3f-b7b8-e084158ba1bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=0973888f233c43e3bf2af18ebd01a932)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:50.940335 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8813841b-45be-4c3f-b7b8-e084158ba1bf admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:51.230808 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8813841b-45be-4c3f-b7b8-e084158ba1bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0973888f233c43e3bf2af18ebd01a932'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:51.232442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8813841b-45be-4c3f-b7b8-e084158ba1bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0973888f233c43e3bf2af18ebd01a932'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:51.242215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8813841b-45be-4c3f-b7b8-e084158ba1bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0973888f233c43e3bf2af18ebd01a932'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:51.361270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8813841b-45be-4c3f-b7b8-e084158ba1bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0973888f233c43e3bf2af18ebd01a932'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:51.362264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-8813841b-45be-4c3f-b7b8-e084158ba1bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0973888f233c43e3bf2af18ebd01a932'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:51.387083 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1028/2039] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:35:50 2022] DELETE /identity/v3/users/0973888f233c43e3bf2af18ebd01a932 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:51.392216 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfdd4d-6f67-480c-8c5a-bacf4b9bb895 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:51.392787 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfdd4d-6f67-480c-8c5a-bacf4b9bb895 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:51.392787 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dfdd4d-6f67-480c-8c5a-bacf4b9bb895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:51.412998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d7dfdd4d-6f67-480c-8c5a-bacf4b9bb895 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:51.414250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d7dfdd4d-6f67-480c-8c5a-bacf4b9bb895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:51.439665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:51.492029 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1012/2040] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:51.504735 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813841b-45be-4c3f-b7b8-e084158ba1bf admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:51.505268 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8813841b-45be-4c3f-b7b8-e084158ba1bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:51.557042 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:51.574429 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813841b-45be-4c3f-b7b8-e084158ba1bf admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:51.578922 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be6190e-8061-4a79-ad43-3cb6c6c87d72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:51.579594 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6190e-8061-4a79-ad43-3cb6c6c87d72 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:51.579875 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6190e-8061-4a79-ad43-3cb6c6c87d72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:51.580198 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6190e-8061-4a79-ad43-3cb6c6c87d72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:51.589247 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be6190e-8061-4a79-ad43-3cb6c6c87d72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:51.590621 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be6190e-8061-4a79-ad43-3cb6c6c87d72 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:51.601084 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1029/2041] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:35:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:35:52.218337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dfdd4d-6f67-480c-8c5a-bacf4b9bb895 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:52.219817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d7dfdd4d-6f67-480c-8c5a-bacf4b9bb895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:52.233844 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dfdd4d-6f67-480c-8c5a-bacf4b9bb895 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:52.239070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be6190e-8061-4a79-ad43-3cb6c6c87d72 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:52.241280 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6be6190e-8061-4a79-ad43-3cb6c6c87d72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:52.251173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c271c5-e51b-4bc1-9ae7-a045e1c04e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:52.252700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c271c5-e51b-4bc1-9ae7-a045e1c04e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:52.252963 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c271c5-e51b-4bc1-9ae7-a045e1c04e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:52.253244 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c271c5-e51b-4bc1-9ae7-a045e1c04e70 admin admin] PATH_INFO: `/v3/projects/28f8719c5d4946fd87ba824631d5b580` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:52.255174 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c271c5-e51b-4bc1-9ae7-a045e1c04e70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28f8719c5d4946fd87ba824631d5b580)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:52.257238 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c271c5-e51b-4bc1-9ae7-a045e1c04e70 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:52.271171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be6190e-8061-4a79-ad43-3cb6c6c87d72 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:52.279373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c648f-f992-48d2-b4a7-386116d2d777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:52.280352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c648f-f992-48d2-b4a7-386116d2d777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:52.283317 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c648f-f992-48d2-b4a7-386116d2d777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:52.283317 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c648f-f992-48d2-b4a7-386116d2d777 admin admin] PATH_INFO: `/v3/users/76b0aa02f6af48e691291fcb4c0da5b3` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:52.283317 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c648f-f992-48d2-b4a7-386116d2d777 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76b0aa02f6af48e691291fcb4c0da5b3)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:52.284549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c648f-f992-48d2-b4a7-386116d2d777 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:52.608828 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ab3c648f-f992-48d2-b4a7-386116d2d777 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76b0aa02f6af48e691291fcb4c0da5b3'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:52.608828 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ab3c648f-f992-48d2-b4a7-386116d2d777 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76b0aa02f6af48e691291fcb4c0da5b3'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:52.609010 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-27c271c5-e51b-4bc1-9ae7-a045e1c04e70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28f8719c5d4946fd87ba824631d5b580', 'request_id': 'req-27c271c5-e51b-4bc1-9ae7-a045e1c04e70'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:52.609438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ab3c648f-f992-48d2-b4a7-386116d2d777 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76b0aa02f6af48e691291fcb4c0da5b3'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:52.609526 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-27c271c5-e51b-4bc1-9ae7-a045e1c04e70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28f8719c5d4946fd87ba824631d5b580', 'request_id': 'req-27c271c5-e51b-4bc1-9ae7-a045e1c04e70'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:52.609526 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-27c271c5-e51b-4bc1-9ae7-a045e1c04e70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28f8719c5d4946fd87ba824631d5b580', 'request_id': 'req-27c271c5-e51b-4bc1-9ae7-a045e1c04e70'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:52.793056 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ab3c648f-f992-48d2-b4a7-386116d2d777 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76b0aa02f6af48e691291fcb4c0da5b3'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:52.793644 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ab3c648f-f992-48d2-b4a7-386116d2d777 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76b0aa02f6af48e691291fcb4c0da5b3'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:52.815534 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1030/2042] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:35:52 2022] DELETE /identity/v3/users/76b0aa02f6af48e691291fcb4c0da5b3 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:35:52.820040 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddb891-d2ed-4c59-9952-df8892ab39d4 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:52.820189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddb891-d2ed-4c59-9952-df8892ab39d4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:52.820455 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddb891-d2ed-4c59-9952-df8892ab39d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:52.838974 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f3ddb891-d2ed-4c59-9952-df8892ab39d4 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:52.840240 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f3ddb891-d2ed-4c59-9952-df8892ab39d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:52.915984 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1031/2043] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:35:52.927996 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddb891-d2ed-4c59-9952-df8892ab39d4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:52.929643 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f3ddb891-d2ed-4c59-9952-df8892ab39d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:52.955455 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-27c271c5-e51b-4bc1-9ae7-a045e1c04e70 admin admin] The token cache is being invalidate because project 28f8719c5d4946fd87ba824631d5b580 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:35:52.955777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-27c271c5-e51b-4bc1-9ae7-a045e1c04e70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:52.956445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-27c271c5-e51b-4bc1-9ae7-a045e1c04e70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:52.958032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1013/2044] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:35:52 2022] DELETE /identity/v3/projects/28f8719c5d4946fd87ba824631d5b580 => generated 0 bytes in 741 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:35:52.970708 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21b27e-a7d1-4a75-8e39-8806b1af2882 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:52.971098 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21b27e-a7d1-4a75-8e39-8806b1af2882 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:52.971381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21b27e-a7d1-4a75-8e39-8806b1af2882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:52.989370 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddb891-d2ed-4c59-9952-df8892ab39d4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:52.991265 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-df21b27e-a7d1-4a75-8e39-8806b1af2882 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:52.992538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-df21b27e-a7d1-4a75-8e39-8806b1af2882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:52.993391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220d6b7-4376-40ff-8d56-69743ed633e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:52.999127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220d6b7-4376-40ff-8d56-69743ed633e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:52.999127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220d6b7-4376-40ff-8d56-69743ed633e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:52.999127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220d6b7-4376-40ff-8d56-69743ed633e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:53.001099 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8220d6b7-4376-40ff-8d56-69743ed633e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:53.023521 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1014/2045] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:35:53.023744 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8220d6b7-4376-40ff-8d56-69743ed633e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:53.025273 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8220d6b7-4376-40ff-8d56-69743ed633e4 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:53.032577 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:53.033071 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d440cf8-29c4-4734-bc5f-384f0f74f2b8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:53.033518 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d440cf8-29c4-4734-bc5f-384f0f74f2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:53.033820 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d440cf8-29c4-4734-bc5f-384f0f74f2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:53.041855 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1032/2046] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:35:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:35:53.063086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9d440cf8-29c4-4734-bc5f-384f0f74f2b8 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:53.064823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9d440cf8-29c4-4734-bc5f-384f0f74f2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:53.083937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1015/2047] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:35:53.085575 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220d6b7-4376-40ff-8d56-69743ed633e4 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:53.098982 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8220d6b7-4376-40ff-8d56-69743ed633e4 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:53.105136 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05c076-278b-49c9-bd1d-21e94cda2946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:53.105984 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05c076-278b-49c9-bd1d-21e94cda2946 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:53.106434 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05c076-278b-49c9-bd1d-21e94cda2946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:53.106834 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05c076-278b-49c9-bd1d-21e94cda2946 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:53.108018 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05c076-278b-49c9-bd1d-21e94cda2946 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:35:53.108448 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05c076-278b-49c9-bd1d-21e94cda2946 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:53.110277 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05c076-278b-49c9-bd1d-21e94cda2946 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:53.129555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1033/2048] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:35:53.135733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d440cf8-29c4-4734-bc5f-384f0f74f2b8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:53.149271 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d440cf8-29c4-4734-bc5f-384f0f74f2b8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:53.154772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c985d-4cc9-4b24-848f-53789f30f096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:53.155805 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c985d-4cc9-4b24-848f-53789f30f096 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:53.156090 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c985d-4cc9-4b24-848f-53789f30f096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:53.156375 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429c985d-4cc9-4b24-848f-53789f30f096 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:53.157276 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c985d-4cc9-4b24-848f-53789f30f096 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:53.159245 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429c985d-4cc9-4b24-848f-53789f30f096 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:53.306268 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1016/2049] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:35:53 2022] POST /identity/v3/projects => generated 381 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:35:53.311674 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05c076-278b-49c9-bd1d-21e94cda2946 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:53.322638 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05c076-278b-49c9-bd1d-21e94cda2946 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:53.384996 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e98c4d-94b4-48ed-ab40-108ec13e20ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:53.385957 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e98c4d-94b4-48ed-ab40-108ec13e20ff admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:53.391578 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e98c4d-94b4-48ed-ab40-108ec13e20ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:53.391578 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e98c4d-94b4-48ed-ab40-108ec13e20ff admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:53.391578 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e98c4d-94b4-48ed-ab40-108ec13e20ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:53.391578 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e98c4d-94b4-48ed-ab40-108ec13e20ff admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:53.490793 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1034/2050] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:35:53 2022] POST /identity/v3/users => generated 291 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:35:53.494595 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c985d-4cc9-4b24-848f-53789f30f096 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:53.506679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429c985d-4cc9-4b24-848f-53789f30f096 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:53.511437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737685d0-395b-4e0f-af02-6f97c93e3907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:53.512628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737685d0-395b-4e0f-af02-6f97c93e3907 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:53.512628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737685d0-395b-4e0f-af02-6f97c93e3907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:53.512920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737685d0-395b-4e0f-af02-6f97c93e3907 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:53.514241 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737685d0-395b-4e0f-af02-6f97c93e3907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:53.516753 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737685d0-395b-4e0f-af02-6f97c93e3907 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:53.529712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1017/2051] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:35:53.534154 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e98c4d-94b4-48ed-ab40-108ec13e20ff admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:53.546466 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e98c4d-94b4-48ed-ab40-108ec13e20ff admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:53.564042 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af71160-95e2-4396-8ec2-dfa331138fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:53.564042 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af71160-95e2-4396-8ec2-dfa331138fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:53.564042 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af71160-95e2-4396-8ec2-dfa331138fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:53.564042 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af71160-95e2-4396-8ec2-dfa331138fa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:53.564593 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af71160-95e2-4396-8ec2-dfa331138fa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:53.568174 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af71160-95e2-4396-8ec2-dfa331138fa9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:53.581033 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1035/2052] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:35:53.585614 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737685d0-395b-4e0f-af02-6f97c93e3907 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:53.598203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737685d0-395b-4e0f-af02-6f97c93e3907 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:53.603681 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec2d2d-000e-48ec-b1fe-fdc7dfd43d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:53.604603 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec2d2d-000e-48ec-b1fe-fdc7dfd43d55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:53.604884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec2d2d-000e-48ec-b1fe-fdc7dfd43d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:53.605159 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec2d2d-000e-48ec-b1fe-fdc7dfd43d55 admin admin] PATH_INFO: `/v3/projects/4df4c3aa7c8044dcbd46064e0e495a84/users/96d7af5479ce46b99e333d91128b826e/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:53.622287 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec2d2d-000e-48ec-b1fe-fdc7dfd43d55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4df4c3aa7c8044dcbd46064e0e495a84, user_id=96d7af5479ce46b99e333d91128b826e, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:53.625710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec2d2d-000e-48ec-b1fe-fdc7dfd43d55 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:53.821772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1018/2053] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:35:53 2022] PUT /identity/v3/projects/4df4c3aa7c8044dcbd46064e0e495a84/users/96d7af5479ce46b99e333d91128b826e/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:53.821956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af71160-95e2-4396-8ec2-dfa331138fa9 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:53.821956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7af71160-95e2-4396-8ec2-dfa331138fa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:53.826854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6192c-5a19-4b64-b184-131e84122ef6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:53.827150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6192c-5a19-4b64-b184-131e84122ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:53.827278 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af71160-95e2-4396-8ec2-dfa331138fa9 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:53.827504 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6192c-5a19-4b64-b184-131e84122ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:53.846575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-98e6192c-5a19-4b64-b184-131e84122ef6 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:53.850525 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-98e6192c-5a19-4b64-b184-131e84122ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:53.887155 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:53.912969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b372539-8519-46bd-8b8d-6ed7c2bdbe45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:53.913867 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b372539-8519-46bd-8b8d-6ed7c2bdbe45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:53.914144 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b372539-8519-46bd-8b8d-6ed7c2bdbe45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:53.914419 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b372539-8519-46bd-8b8d-6ed7c2bdbe45 admin admin] PATH_INFO: `/v3/projects/10f2fa56d4ac4ba2839736e58b55e8ff` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:53.917324 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b372539-8519-46bd-8b8d-6ed7c2bdbe45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10f2fa56d4ac4ba2839736e58b55e8ff)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:53.917610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b372539-8519-46bd-8b8d-6ed7c2bdbe45 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:53.921465 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1019/2054] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:53.931201 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6192c-5a19-4b64-b184-131e84122ef6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:53.933086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-98e6192c-5a19-4b64-b184-131e84122ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:53.992198 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6192c-5a19-4b64-b184-131e84122ef6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:53.996125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c41348-03b7-425e-8d71-1d29de6580d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:53.997326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c41348-03b7-425e-8d71-1d29de6580d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:53.998225 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c41348-03b7-425e-8d71-1d29de6580d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:53.998488 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c41348-03b7-425e-8d71-1d29de6580d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:54.018623 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c41348-03b7-425e-8d71-1d29de6580d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:54.020208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c41348-03b7-425e-8d71-1d29de6580d3 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:54.031427 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1020/2055] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:35:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:35:54.050740 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9b372539-8519-46bd-8b8d-6ed7c2bdbe45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10f2fa56d4ac4ba2839736e58b55e8ff', 'request_id': 'req-9b372539-8519-46bd-8b8d-6ed7c2bdbe45'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:54.130796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9b372539-8519-46bd-8b8d-6ed7c2bdbe45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10f2fa56d4ac4ba2839736e58b55e8ff', 'request_id': 'req-9b372539-8519-46bd-8b8d-6ed7c2bdbe45'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:54.141099 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9b372539-8519-46bd-8b8d-6ed7c2bdbe45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10f2fa56d4ac4ba2839736e58b55e8ff', 'request_id': 'req-9b372539-8519-46bd-8b8d-6ed7c2bdbe45'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:54.148924 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:54.161004 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9b372539-8519-46bd-8b8d-6ed7c2bdbe45 admin admin] The token cache is being invalidate because project 10f2fa56d4ac4ba2839736e58b55e8ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:35:54.161590 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9b372539-8519-46bd-8b8d-6ed7c2bdbe45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:54.162631 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9b372539-8519-46bd-8b8d-6ed7c2bdbe45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:54.169101 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1036/2056] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:35:53 2022] DELETE /identity/v3/projects/10f2fa56d4ac4ba2839736e58b55e8ff => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:54.175575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a3606-0544-48d3-909d-0585c4da00c5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:54.175575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a3606-0544-48d3-909d-0585c4da00c5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:54.175575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a3606-0544-48d3-909d-0585c4da00c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:54.200570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-112a3606-0544-48d3-909d-0585c4da00c5 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:54.202864 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-112a3606-0544-48d3-909d-0585c4da00c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:54.286277 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1021/2057] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:54.289052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e946630-caac-41e0-abb0-5a1a6dc218a4 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:54.289052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e946630-caac-41e0-abb0-5a1a6dc218a4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:54.289052 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e946630-caac-41e0-abb0-5a1a6dc218a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:54.309373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3e946630-caac-41e0-abb0-5a1a6dc218a4 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:54.310861 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3e946630-caac-41e0-abb0-5a1a6dc218a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:54.320626 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1037/2058] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:54.324526 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a3606-0544-48d3-909d-0585c4da00c5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:54.335521 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a3606-0544-48d3-909d-0585c4da00c5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:54.341297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e5d99-ff32-4eef-ab78-7ee86294d150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:54.342714 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e5d99-ff32-4eef-ab78-7ee86294d150 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:54.343002 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e5d99-ff32-4eef-ab78-7ee86294d150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:54.343311 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e5d99-ff32-4eef-ab78-7ee86294d150 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:54.344381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e5d99-ff32-4eef-ab78-7ee86294d150 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:35:54.344805 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e5d99-ff32-4eef-ab78-7ee86294d150 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:54.347131 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e5d99-ff32-4eef-ab78-7ee86294d150 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:54.362770 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1022/2059] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:35:54.367313 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e946630-caac-41e0-abb0-5a1a6dc218a4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:54.378807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e946630-caac-41e0-abb0-5a1a6dc218a4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:54.393477 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c7887-2b54-46c9-b7d1-2e7c753abdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:54.394171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c7887-2b54-46c9-b7d1-2e7c753abdb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:54.394457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c7887-2b54-46c9-b7d1-2e7c753abdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:54.394738 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c7887-2b54-46c9-b7d1-2e7c753abdb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:54.395520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c7887-2b54-46c9-b7d1-2e7c753abdb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:54.397670 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c7887-2b54-46c9-b7d1-2e7c753abdb4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:54.525787 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1038/2060] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:35:54 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:35:54.526973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e5d99-ff32-4eef-ab78-7ee86294d150 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:54.540563 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e5d99-ff32-4eef-ab78-7ee86294d150 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:54.617790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd7731-a1b8-4d9e-a062-acee24f69f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:54.618562 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd7731-a1b8-4d9e-a062-acee24f69f95 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:54.619004 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd7731-a1b8-4d9e-a062-acee24f69f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:54.619290 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd7731-a1b8-4d9e-a062-acee24f69f95 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:54.620339 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd7731-a1b8-4d9e-a062-acee24f69f95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:54.622345 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd7731-a1b8-4d9e-a062-acee24f69f95 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:54.728757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1023/2061] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:35:54 2022] POST /identity/v3/users => generated 292 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:35:54.733435 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c7887-2b54-46c9-b7d1-2e7c753abdb4 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:54.743952 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c7887-2b54-46c9-b7d1-2e7c753abdb4 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:54.748795 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35889f29-a4dd-4c3e-a67f-131f3f46f7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:54.749463 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35889f29-a4dd-4c3e-a67f-131f3f46f7b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:54.749748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35889f29-a4dd-4c3e-a67f-131f3f46f7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:54.750025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35889f29-a4dd-4c3e-a67f-131f3f46f7b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:54.750773 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35889f29-a4dd-4c3e-a67f-131f3f46f7b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:54.752440 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35889f29-a4dd-4c3e-a67f-131f3f46f7b6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:54.765911 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1039/2062] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:35:54.770328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd7731-a1b8-4d9e-a062-acee24f69f95 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:54.782525 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd7731-a1b8-4d9e-a062-acee24f69f95 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:54.787817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b58f994-9b70-4846-acc2-d0b2ac42a427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:54.788609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58f994-9b70-4846-acc2-d0b2ac42a427 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:54.792503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58f994-9b70-4846-acc2-d0b2ac42a427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:54.792503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58f994-9b70-4846-acc2-d0b2ac42a427 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:54.792503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58f994-9b70-4846-acc2-d0b2ac42a427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:54.792503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58f994-9b70-4846-acc2-d0b2ac42a427 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:54.804486 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1024/2063] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:35:54.808728 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35889f29-a4dd-4c3e-a67f-131f3f46f7b6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:54.821062 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35889f29-a4dd-4c3e-a67f-131f3f46f7b6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:54.828976 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3472a8d3-553b-4fea-938b-067dd3b31e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:54.830028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472a8d3-553b-4fea-938b-067dd3b31e7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:54.830440 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472a8d3-553b-4fea-938b-067dd3b31e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:54.830779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472a8d3-553b-4fea-938b-067dd3b31e7b admin admin] PATH_INFO: `/v3/projects/16cee814ae7e491b8c4c4f880f17e1ba/users/3bd60029480046e4a7730d0e1a1dd892/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:54.843750 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b58f994-9b70-4846-acc2-d0b2ac42a427 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:54.845610 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4b58f994-9b70-4846-acc2-d0b2ac42a427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:54.853335 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472a8d3-553b-4fea-938b-067dd3b31e7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=16cee814ae7e491b8c4c4f880f17e1ba, user_id=3bd60029480046e4a7730d0e1a1dd892, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:54.856511 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3472a8d3-553b-4fea-938b-067dd3b31e7b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:54.860113 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b58f994-9b70-4846-acc2-d0b2ac42a427 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:54.865497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9790e9-ddb8-4e6f-aa25-b48fc55bff18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:54.866492 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9790e9-ddb8-4e6f-aa25-b48fc55bff18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:54.866738 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9790e9-ddb8-4e6f-aa25-b48fc55bff18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:54.867018 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9790e9-ddb8-4e6f-aa25-b48fc55bff18 admin admin] PATH_INFO: `/v3/users/1be16beb38474ae7b9f4b811552823c9` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:54.868346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9790e9-ddb8-4e6f-aa25-b48fc55bff18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1be16beb38474ae7b9f4b811552823c9)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:54.870439 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9790e9-ddb8-4e6f-aa25-b48fc55bff18 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:54.988834 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1040/2064] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:35:54 2022] PUT /identity/v3/projects/16cee814ae7e491b8c4c4f880f17e1ba/users/3bd60029480046e4a7730d0e1a1dd892/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:35:55.000631 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb43ee-3685-468d-93e4-2278a5f789b2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:55.000932 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb43ee-3685-468d-93e4-2278a5f789b2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:55.001277 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb43ee-3685-468d-93e4-2278a5f789b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:55.040165 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d7eb43ee-3685-468d-93e4-2278a5f789b2 None None] MFA Rules not processed for user `3bd60029480046e4a7730d0e1a1dd892`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:55.043447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d7eb43ee-3685-468d-93e4-2278a5f789b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:55.120745 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3f9790e9-ddb8-4e6f-aa25-b48fc55bff18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1be16beb38474ae7b9f4b811552823c9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:55.121400 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3f9790e9-ddb8-4e6f-aa25-b48fc55bff18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1be16beb38474ae7b9f4b811552823c9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:55.151162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1041/2065] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:55.165888 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb43ee-3685-468d-93e4-2278a5f789b2 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:55.169297 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d7eb43ee-3685-468d-93e4-2278a5f789b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:55.255619 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb43ee-3685-468d-93e4-2278a5f789b2 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:55.259945 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ecbbc-6ae0-4a1d-8978-e6ba906d22b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:55.261149 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ecbbc-6ae0-4a1d-8978-e6ba906d22b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:55.261447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ecbbc-6ae0-4a1d-8978-e6ba906d22b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:55.261715 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ecbbc-6ae0-4a1d-8978-e6ba906d22b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:55.273466 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249ecbbc-6ae0-4a1d-8978-e6ba906d22b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:55.274997 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249ecbbc-6ae0-4a1d-8978-e6ba906d22b2 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:55.285780 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1042/2066] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:35:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:35:55.313305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3f9790e9-ddb8-4e6f-aa25-b48fc55bff18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1be16beb38474ae7b9f4b811552823c9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:55.321876 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3f9790e9-ddb8-4e6f-aa25-b48fc55bff18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1be16beb38474ae7b9f4b811552823c9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:55.322644 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3f9790e9-ddb8-4e6f-aa25-b48fc55bff18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1be16beb38474ae7b9f4b811552823c9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:55.345604 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1025/2067] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:35:54 2022] DELETE /identity/v3/users/1be16beb38474ae7b9f4b811552823c9 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:55.351000 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28794ed7-4c23-4e0a-9063-c107458ed83a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:55.351590 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28794ed7-4c23-4e0a-9063-c107458ed83a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:55.351698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:55.352108 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28794ed7-4c23-4e0a-9063-c107458ed83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:55.381119 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-28794ed7-4c23-4e0a-9063-c107458ed83a None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:55.382684 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-28794ed7-4c23-4e0a-9063-c107458ed83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:55.485868 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1043/2068] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:55.520292 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9790e9-ddb8-4e6f-aa25-b48fc55bff18 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:55.521749 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3f9790e9-ddb8-4e6f-aa25-b48fc55bff18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:55.599326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9790e9-ddb8-4e6f-aa25-b48fc55bff18 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:55.603457 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113949d-a044-44c6-9874-6a0f5db72c6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:55.604248 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113949d-a044-44c6-9874-6a0f5db72c6c service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:55.604539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113949d-a044-44c6-9874-6a0f5db72c6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:55.604814 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1113949d-a044-44c6-9874-6a0f5db72c6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:55.615241 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1113949d-a044-44c6-9874-6a0f5db72c6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:55.616713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1113949d-a044-44c6-9874-6a0f5db72c6c service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:55.629680 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1026/2069] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:35:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:35:55.707897 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:56.098473 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8161e3-4a82-411f-9ecf-58a385c08fa4 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:56.099006 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8161e3-4a82-411f-9ecf-58a385c08fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:56.099922 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8161e3-4a82-411f-9ecf-58a385c08fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:56.159266 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1f8161e3-4a82-411f-9ecf-58a385c08fa4 None None] MFA Rules not processed for user `96d7af5479ce46b99e333d91128b826e`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:56.160919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1f8161e3-4a82-411f-9ecf-58a385c08fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:56.225863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113949d-a044-44c6-9874-6a0f5db72c6c service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:56.228250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1113949d-a044-44c6-9874-6a0f5db72c6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:56.261484 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1113949d-a044-44c6-9874-6a0f5db72c6c service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:56.270244 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29b1c3-22d0-4fa3-80cd-3ffe7cc30cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:56.271976 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29b1c3-22d0-4fa3-80cd-3ffe7cc30cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:56.272460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29b1c3-22d0-4fa3-80cd-3ffe7cc30cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:56.272904 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29b1c3-22d0-4fa3-80cd-3ffe7cc30cd0 admin admin] PATH_INFO: `/v3/projects/eddad92f431b4341829cac52d5cf0ed7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:56.275424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29b1c3-22d0-4fa3-80cd-3ffe7cc30cd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eddad92f431b4341829cac52d5cf0ed7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:56.278667 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29b1c3-22d0-4fa3-80cd-3ffe7cc30cd0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:56.284787 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1044/2070] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:56.305322 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8161e3-4a82-411f-9ecf-58a385c08fa4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:56.307205 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1f8161e3-4a82-411f-9ecf-58a385c08fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:56.405455 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8161e3-4a82-411f-9ecf-58a385c08fa4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:56.409446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d5b39-720c-4ba7-bab1-de848130acd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:56.410382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d5b39-720c-4ba7-bab1-de848130acd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:56.410935 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d5b39-720c-4ba7-bab1-de848130acd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:56.412325 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d5b39-720c-4ba7-bab1-de848130acd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:56.423457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d5b39-720c-4ba7-bab1-de848130acd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:56.437921 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d5b39-720c-4ba7-bab1-de848130acd9 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:56.437921 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1045/2071] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:35:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:35:56.499261 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc4438-3104-48f8-a272-821513c61fdd None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:56.499483 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc4438-3104-48f8-a272-821513c61fdd None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:56.499815 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecc4438-3104-48f8-a272-821513c61fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:56.519462 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1ecc4438-3104-48f8-a272-821513c61fdd None None] MFA Rules not processed for user `0a26028451b345968fe86774f44736dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:56.520709 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1ecc4438-3104-48f8-a272-821513c61fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:56.558034 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ae29b1c3-22d0-4fa3-80cd-3ffe7cc30cd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eddad92f431b4341829cac52d5cf0ed7', 'request_id': 'req-ae29b1c3-22d0-4fa3-80cd-3ffe7cc30cd0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:56.567011 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ae29b1c3-22d0-4fa3-80cd-3ffe7cc30cd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eddad92f431b4341829cac52d5cf0ed7', 'request_id': 'req-ae29b1c3-22d0-4fa3-80cd-3ffe7cc30cd0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:56.574221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ae29b1c3-22d0-4fa3-80cd-3ffe7cc30cd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eddad92f431b4341829cac52d5cf0ed7', 'request_id': 'req-ae29b1c3-22d0-4fa3-80cd-3ffe7cc30cd0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:56.626668 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1046/2072] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:35:57.036182 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ae29b1c3-22d0-4fa3-80cd-3ffe7cc30cd0 admin admin] The token cache is being invalidate because project eddad92f431b4341829cac52d5cf0ed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:35:57.036663 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ae29b1c3-22d0-4fa3-80cd-3ffe7cc30cd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:57.037317 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ae29b1c3-22d0-4fa3-80cd-3ffe7cc30cd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:57.041249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1027/2073] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:35:56 2022] DELETE /identity/v3/projects/eddad92f431b4341829cac52d5cf0ed7 => generated 0 bytes in 814 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:57.043694 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc4438-3104-48f8-a272-821513c61fdd None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:57.057447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecc4438-3104-48f8-a272-821513c61fdd None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:57.120173 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41434274-c65d-4d47-974f-fe52027d51bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:57.121260 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41434274-c65d-4d47-974f-fe52027d51bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:57.121667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41434274-c65d-4d47-974f-fe52027d51bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:57.122019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41434274-c65d-4d47-974f-fe52027d51bf admin admin] PATH_INFO: `/v3/users/a66f7549150d4f368239a7090e298a9a` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:57.123813 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41434274-c65d-4d47-974f-fe52027d51bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=a66f7549150d4f368239a7090e298a9a)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:57.125834 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41434274-c65d-4d47-974f-fe52027d51bf admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:57.298602 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:35:57.404287 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-41434274-c65d-4d47-974f-fe52027d51bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a66f7549150d4f368239a7090e298a9a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:57.405485 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-41434274-c65d-4d47-974f-fe52027d51bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a66f7549150d4f368239a7090e298a9a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:57.412953 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-41434274-c65d-4d47-974f-fe52027d51bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a66f7549150d4f368239a7090e298a9a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:57.488045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-41434274-c65d-4d47-974f-fe52027d51bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a66f7549150d4f368239a7090e298a9a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:57.492955 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-41434274-c65d-4d47-974f-fe52027d51bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a66f7549150d4f368239a7090e298a9a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:57.530962 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1047/2074] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:35:57 2022] DELETE /identity/v3/users/a66f7549150d4f368239a7090e298a9a => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:58.165084 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29b1c3-22d0-4fa3-80cd-3ffe7cc30cd0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:58.166716 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ae29b1c3-22d0-4fa3-80cd-3ffe7cc30cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:58.196844 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29b1c3-22d0-4fa3-80cd-3ffe7cc30cd0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:58.265187 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a1b5b8-6a33-4c5d-80b2-f038cfe03660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:58.266071 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1b5b8-6a33-4c5d-80b2-f038cfe03660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:58.266367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1b5b8-6a33-4c5d-80b2-f038cfe03660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:58.266627 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a1b5b8-6a33-4c5d-80b2-f038cfe03660 admin admin] PATH_INFO: `/v3/projects/833a7542256341fd9d4cfac82cf1f9f1` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:58.267891 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a1b5b8-6a33-4c5d-80b2-f038cfe03660 admin admin] RBAC: Authorizing `identity:delete_project(project_id=833a7542256341fd9d4cfac82cf1f9f1)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:58.269821 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a1b5b8-6a33-4c5d-80b2-f038cfe03660 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:58.437756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c8a1b5b8-6a33-4c5d-80b2-f038cfe03660 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '833a7542256341fd9d4cfac82cf1f9f1', 'request_id': 'req-c8a1b5b8-6a33-4c5d-80b2-f038cfe03660'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:58.443943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c8a1b5b8-6a33-4c5d-80b2-f038cfe03660 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '833a7542256341fd9d4cfac82cf1f9f1', 'request_id': 'req-c8a1b5b8-6a33-4c5d-80b2-f038cfe03660'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:58.449423 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c8a1b5b8-6a33-4c5d-80b2-f038cfe03660 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '833a7542256341fd9d4cfac82cf1f9f1', 'request_id': 'req-c8a1b5b8-6a33-4c5d-80b2-f038cfe03660'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:58.598386 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c8a1b5b8-6a33-4c5d-80b2-f038cfe03660 admin admin] The token cache is being invalidate because project 833a7542256341fd9d4cfac82cf1f9f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:35:58.598386 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c8a1b5b8-6a33-4c5d-80b2-f038cfe03660 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:58.598386 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c8a1b5b8-6a33-4c5d-80b2-f038cfe03660 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:35:58.598386 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1028/2075] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:35:58 2022] DELETE /identity/v3/projects/833a7542256341fd9d4cfac82cf1f9f1 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:58.599770 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0165900-04af-46a0-8316-56c5a02969f9 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:58.600231 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0165900-04af-46a0-8316-56c5a02969f9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:58.600408 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0165900-04af-46a0-8316-56c5a02969f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:58.631584 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a0165900-04af-46a0-8316-56c5a02969f9 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:58.632946 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a0165900-04af-46a0-8316-56c5a02969f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:58.686502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4004b-c2b4-4072-a1b1-55b43e878167 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:58.686763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4004b-c2b4-4072-a1b1-55b43e878167 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:58.687251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4004b-c2b4-4072-a1b1-55b43e878167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:58.706071 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1048/2076] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:58.709100 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-92c4004b-c2b4-4072-a1b1-55b43e878167 None None] MFA Rules not processed for user `96d7af5479ce46b99e333d91128b826e`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:58.710485 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-92c4004b-c2b4-4072-a1b1-55b43e878167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:58.716515 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77af7cf-be6a-40fe-9ac1-7b9dd46c2a6a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:58.728866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77af7cf-be6a-40fe-9ac1-7b9dd46c2a6a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:58.728866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77af7cf-be6a-40fe-9ac1-7b9dd46c2a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:58.740848 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c77af7cf-be6a-40fe-9ac1-7b9dd46c2a6a None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:58.742118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c77af7cf-be6a-40fe-9ac1-7b9dd46c2a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:58.753442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1049/2077] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:35:58.758256 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77af7cf-be6a-40fe-9ac1-7b9dd46c2a6a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:58.769415 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77af7cf-be6a-40fe-9ac1-7b9dd46c2a6a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:58.774463 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a69026-3bc7-4b35-91e3-6ddb3d44b533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:58.775137 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a69026-3bc7-4b35-91e3-6ddb3d44b533 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:58.775711 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a69026-3bc7-4b35-91e3-6ddb3d44b533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:58.775822 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a69026-3bc7-4b35-91e3-6ddb3d44b533 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:58.776667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a69026-3bc7-4b35-91e3-6ddb3d44b533 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:35:58.776794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a69026-3bc7-4b35-91e3-6ddb3d44b533 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:58.778403 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a69026-3bc7-4b35-91e3-6ddb3d44b533 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:58.791447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1050/2078] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:35:58.792470 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1029/2079] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:35:58.796271 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a69026-3bc7-4b35-91e3-6ddb3d44b533 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:58.807447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a69026-3bc7-4b35-91e3-6ddb3d44b533 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:58.813437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b61f4-c01e-4858-9221-6da75e85b8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:58.814133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b61f4-c01e-4858-9221-6da75e85b8d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:58.814448 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b61f4-c01e-4858-9221-6da75e85b8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:58.814783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b61f4-c01e-4858-9221-6da75e85b8d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:58.815532 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b61f4-c01e-4858-9221-6da75e85b8d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:58.817586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09b61f4-c01e-4858-9221-6da75e85b8d2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:58.952927 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1051/2080] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:35:58 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:35:58.954554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4004b-c2b4-4072-a1b1-55b43e878167 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:58.968839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4004b-c2b4-4072-a1b1-55b43e878167 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:59.052079 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534535b-ca72-426b-83b5-906cb2bfe72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:59.057593 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534535b-ca72-426b-83b5-906cb2bfe72e admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:59.057593 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534535b-ca72-426b-83b5-906cb2bfe72e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:59.057593 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9534535b-ca72-426b-83b5-906cb2bfe72e admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:59.057593 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534535b-ca72-426b-83b5-906cb2bfe72e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:59.057593 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9534535b-ca72-426b-83b5-906cb2bfe72e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:59.195620 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1030/2081] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:35:58 2022] POST /identity/v3/users => generated 290 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:35:59.199833 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b61f4-c01e-4858-9221-6da75e85b8d2 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:59.224485 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b61f4-c01e-4858-9221-6da75e85b8d2 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:59.249764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e77dc-3b61-4fa0-866a-d20ae4146bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:59.252589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e77dc-3b61-4fa0-866a-d20ae4146bce admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:59.256015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e77dc-3b61-4fa0-866a-d20ae4146bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:59.256310 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e77dc-3b61-4fa0-866a-d20ae4146bce admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:59.257169 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e77dc-3b61-4fa0-866a-d20ae4146bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:59.258960 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e77dc-3b61-4fa0-866a-d20ae4146bce admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:59.273684 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1052/2082] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:35:59.278033 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534535b-ca72-426b-83b5-906cb2bfe72e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:59.289218 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9534535b-ca72-426b-83b5-906cb2bfe72e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:59.293970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a4cbf7-59ae-4760-97da-a4320028c196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:59.294646 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4cbf7-59ae-4760-97da-a4320028c196 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:59.295650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4cbf7-59ae-4760-97da-a4320028c196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:59.295981 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4cbf7-59ae-4760-97da-a4320028c196 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:59.297203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a4cbf7-59ae-4760-97da-a4320028c196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:59.298846 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a4cbf7-59ae-4760-97da-a4320028c196 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:59.309876 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1031/2083] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:35:59.314953 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e77dc-3b61-4fa0-866a-d20ae4146bce admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:59.331724 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e77dc-3b61-4fa0-866a-d20ae4146bce admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:59.336415 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db35ce-8d01-4c7b-9c8e-c36d61f91afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:59.337420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db35ce-8d01-4c7b-9c8e-c36d61f91afe admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:59.337750 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db35ce-8d01-4c7b-9c8e-c36d61f91afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:59.338031 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db35ce-8d01-4c7b-9c8e-c36d61f91afe admin admin] PATH_INFO: `/v3/projects/67e4bbb83c0b4f4585b0d0afd6e6a35e/users/362a81f2165c4376a0cc4041e037253c/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:59.362658 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db35ce-8d01-4c7b-9c8e-c36d61f91afe admin admin] RBAC: Authorizing `identity:create_grant(project_id=67e4bbb83c0b4f4585b0d0afd6e6a35e, user_id=362a81f2165c4376a0cc4041e037253c, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:59.365902 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db35ce-8d01-4c7b-9c8e-c36d61f91afe admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:59.501968 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1053/2084] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:35:59 2022] PUT /identity/v3/projects/67e4bbb83c0b4f4585b0d0afd6e6a35e/users/362a81f2165c4376a0cc4041e037253c/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:35:59.513652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7b37f-a4d8-4de9-8976-c49f436398e7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:59.514346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7b37f-a4d8-4de9-8976-c49f436398e7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:59.514870 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7b37f-a4d8-4de9-8976-c49f436398e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:59.566784 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4cd7b37f-a4d8-4de9-8976-c49f436398e7 None None] MFA Rules not processed for user `362a81f2165c4376a0cc4041e037253c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:35:59.568816 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4cd7b37f-a4d8-4de9-8976-c49f436398e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:35:59.681070 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1032/2085] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:35:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:35:59.699688 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db35ce-8d01-4c7b-9c8e-c36d61f91afe admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:59.794409 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db35ce-8d01-4c7b-9c8e-c36d61f91afe admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:35:59.799695 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1277f23b-6911-4e7f-b673-75769e6ae98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:35:59.802603 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277f23b-6911-4e7f-b673-75769e6ae98b admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:35:59.803176 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277f23b-6911-4e7f-b673-75769e6ae98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:35:59.803515 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1277f23b-6911-4e7f-b673-75769e6ae98b admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:35:59.804428 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1277f23b-6911-4e7f-b673-75769e6ae98b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:35:59.806534 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1277f23b-6911-4e7f-b673-75769e6ae98b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:35:59.975752 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1054/2086] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:35:59 2022] POST /identity/v3/projects => generated 377 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:35:59.979763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7b37f-a4d8-4de9-8976-c49f436398e7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:35:59.990552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7b37f-a4d8-4de9-8976-c49f436398e7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:00.048150 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:00.052304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b84711-c773-4837-8f68-79afc4709069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:00.053155 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b84711-c773-4837-8f68-79afc4709069 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:00.053588 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b84711-c773-4837-8f68-79afc4709069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:00.053884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b84711-c773-4837-8f68-79afc4709069 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:00.054753 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b84711-c773-4837-8f68-79afc4709069 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:00.056747 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b84711-c773-4837-8f68-79afc4709069 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:00.254198 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1033/2087] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:35:59 2022] POST /identity/v3/users => generated 289 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:36:00.258219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1277f23b-6911-4e7f-b673-75769e6ae98b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:00.269585 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1277f23b-6911-4e7f-b673-75769e6ae98b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:00.274197 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcaf58d-a9bb-41bd-ac25-c5dad7e1155f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:00.274857 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcaf58d-a9bb-41bd-ac25-c5dad7e1155f admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:00.275145 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcaf58d-a9bb-41bd-ac25-c5dad7e1155f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:00.275426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcaf58d-a9bb-41bd-ac25-c5dad7e1155f admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:00.276222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcaf58d-a9bb-41bd-ac25-c5dad7e1155f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:00.277844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcaf58d-a9bb-41bd-ac25-c5dad7e1155f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:00.289189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1055/2088] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:36:00.293306 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b84711-c773-4837-8f68-79afc4709069 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:00.303747 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b84711-c773-4837-8f68-79afc4709069 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:00.308468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abae739a-f1fe-4d4f-ae8f-6b43ec7c14b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:00.309065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae739a-f1fe-4d4f-ae8f-6b43ec7c14b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:00.309348 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae739a-f1fe-4d4f-ae8f-6b43ec7c14b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:00.309692 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abae739a-f1fe-4d4f-ae8f-6b43ec7c14b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:00.310356 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae739a-f1fe-4d4f-ae8f-6b43ec7c14b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:00.312733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abae739a-f1fe-4d4f-ae8f-6b43ec7c14b1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:00.324203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1034/2089] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:36:00.334665 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcaf58d-a9bb-41bd-ac25-c5dad7e1155f admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:00.345107 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcaf58d-a9bb-41bd-ac25-c5dad7e1155f admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:00.349729 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce52cf-5b42-4f48-9ea7-9c0facdafae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:00.350610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce52cf-5b42-4f48-9ea7-9c0facdafae3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:00.350895 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce52cf-5b42-4f48-9ea7-9c0facdafae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:00.351173 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce52cf-5b42-4f48-9ea7-9c0facdafae3 admin admin] PATH_INFO: `/v3/projects/4b8b725a6ef4466fbe50bf3bf7073dc2/users/eb199dc2a4424567b469e56d3785aec0/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:00.366587 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dce52cf-5b42-4f48-9ea7-9c0facdafae3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b8b725a6ef4466fbe50bf3bf7073dc2, user_id=eb199dc2a4424567b469e56d3785aec0, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:00.369313 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dce52cf-5b42-4f48-9ea7-9c0facdafae3 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:00.483833 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1056/2090] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:36:00 2022] PUT /identity/v3/projects/4b8b725a6ef4466fbe50bf3bf7073dc2/users/eb199dc2a4424567b469e56d3785aec0/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:00.496533 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfb32b-4dba-4598-ad2e-652524ec6d75 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:00.497056 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfb32b-4dba-4598-ad2e-652524ec6d75 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:00.497517 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfb32b-4dba-4598-ad2e-652524ec6d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:00.533869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-21bfb32b-4dba-4598-ad2e-652524ec6d75 None None] MFA Rules not processed for user `eb199dc2a4424567b469e56d3785aec0`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:00.533869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-21bfb32b-4dba-4598-ad2e-652524ec6d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:00.642233 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1035/2091] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:00.647823 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce52cf-5b42-4f48-9ea7-9c0facdafae3 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:00.736664 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce52cf-5b42-4f48-9ea7-9c0facdafae3 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:00.742882 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a97de2-6ade-4bee-97a0-321d5bfd8006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:00.743619 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a97de2-6ade-4bee-97a0-321d5bfd8006 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:00.743904 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a97de2-6ade-4bee-97a0-321d5bfd8006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:00.744187 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a97de2-6ade-4bee-97a0-321d5bfd8006 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:00.744899 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a97de2-6ade-4bee-97a0-321d5bfd8006 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:00.746887 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a97de2-6ade-4bee-97a0-321d5bfd8006 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:00.935123 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1057/2092] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:36:00 2022] POST /identity/v3/projects => generated 379 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:36:00.939738 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfb32b-4dba-4598-ad2e-652524ec6d75 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:00.961930 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfb32b-4dba-4598-ad2e-652524ec6d75 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:01.049566 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59391ad2-107f-4c04-9b02-7a596782f036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:01.050373 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59391ad2-107f-4c04-9b02-7a596782f036 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:01.050726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59391ad2-107f-4c04-9b02-7a596782f036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:01.051079 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59391ad2-107f-4c04-9b02-7a596782f036 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:01.051921 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59391ad2-107f-4c04-9b02-7a596782f036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:01.053910 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59391ad2-107f-4c04-9b02-7a596782f036 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:01.162951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1036/2093] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:36:00 2022] POST /identity/v3/users => generated 296 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:36:01.167446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a97de2-6ade-4bee-97a0-321d5bfd8006 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:01.181174 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a97de2-6ade-4bee-97a0-321d5bfd8006 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:01.195689 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998be5b-936b-482a-a3b3-c0503348f696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:01.196492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998be5b-936b-482a-a3b3-c0503348f696 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:01.196832 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998be5b-936b-482a-a3b3-c0503348f696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:01.197155 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f998be5b-936b-482a-a3b3-c0503348f696 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:01.197985 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998be5b-936b-482a-a3b3-c0503348f696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:01.200056 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f998be5b-936b-482a-a3b3-c0503348f696 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:01.213346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1058/2094] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:36:01.219086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59391ad2-107f-4c04-9b02-7a596782f036 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:01.230977 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59391ad2-107f-4c04-9b02-7a596782f036 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:01.238062 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2fb46-298a-4431-afeb-48b7f7fe2815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:01.239093 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2fb46-298a-4431-afeb-48b7f7fe2815 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:01.239493 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2fb46-298a-4431-afeb-48b7f7fe2815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:01.239892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2fb46-298a-4431-afeb-48b7f7fe2815 admin admin] PATH_INFO: `/v3/projects/e7ab1b5c8ad64c108c41149d7b33cd5a/users/f61d0c1c0ddc45849468e72ce734df4c/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:01.259866 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2fb46-298a-4431-afeb-48b7f7fe2815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7ab1b5c8ad64c108c41149d7b33cd5a, user_id=f61d0c1c0ddc45849468e72ce734df4c, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:01.262803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2fb46-298a-4431-afeb-48b7f7fe2815 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:01.393664 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1037/2095] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:36:01 2022] PUT /identity/v3/projects/e7ab1b5c8ad64c108c41149d7b33cd5a/users/f61d0c1c0ddc45849468e72ce734df4c/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:01.397844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998be5b-936b-482a-a3b3-c0503348f696 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:01.406927 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:01.418324 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f998be5b-936b-482a-a3b3-c0503348f696 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:01.520399 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05df5bb-bcbb-402f-9cee-fbea8a5ca844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:01.521538 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05df5bb-bcbb-402f-9cee-fbea8a5ca844 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:01.522109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05df5bb-bcbb-402f-9cee-fbea8a5ca844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:01.522692 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05df5bb-bcbb-402f-9cee-fbea8a5ca844 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:01.524169 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05df5bb-bcbb-402f-9cee-fbea8a5ca844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:01.527601 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05df5bb-bcbb-402f-9cee-fbea8a5ca844 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:01.553478 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1059/2096] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:36:01.557473 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2fb46-298a-4431-afeb-48b7f7fe2815 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:01.570705 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2fb46-298a-4431-afeb-48b7f7fe2815 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:01.578823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533dd288-4fc2-47e8-82e6-8ff7d3962335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:01.579892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533dd288-4fc2-47e8-82e6-8ff7d3962335 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:01.580540 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533dd288-4fc2-47e8-82e6-8ff7d3962335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:01.581072 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533dd288-4fc2-47e8-82e6-8ff7d3962335 admin admin] PATH_INFO: `/v3/projects/e7ab1b5c8ad64c108c41149d7b33cd5a/users/f61d0c1c0ddc45849468e72ce734df4c/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:01.584479 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533dd288-4fc2-47e8-82e6-8ff7d3962335 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7ab1b5c8ad64c108c41149d7b33cd5a, user_id=f61d0c1c0ddc45849468e72ce734df4c, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:01.589774 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533dd288-4fc2-47e8-82e6-8ff7d3962335 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:01.608310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1038/2097] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:36:01 2022] PUT /identity/v3/projects/e7ab1b5c8ad64c108c41149d7b33cd5a/users/f61d0c1c0ddc45849468e72ce734df4c/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:01.621135 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fcce5-9c03-4048-a5dd-36f7a877648c None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:01.621591 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fcce5-9c03-4048-a5dd-36f7a877648c None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:01.625278 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7fcce5-9c03-4048-a5dd-36f7a877648c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:01.659604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2d7fcce5-9c03-4048-a5dd-36f7a877648c None None] MFA Rules not processed for user `f61d0c1c0ddc45849468e72ce734df4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:01.660796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2d7fcce5-9c03-4048-a5dd-36f7a877648c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:01.680329 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:01.754478 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1060/2098] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:01 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:01.770252 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533dd288-4fc2-47e8-82e6-8ff7d3962335 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:01.772111 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-533dd288-4fc2-47e8-82e6-8ff7d3962335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:01.862371 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533dd288-4fc2-47e8-82e6-8ff7d3962335 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:01.866819 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c3884-af3b-4941-bcad-13961f6d2c6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:01.867534 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c3884-af3b-4941-bcad-13961f6d2c6f service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:01.867822 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c3884-af3b-4941-bcad-13961f6d2c6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:01.868099 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c3884-af3b-4941-bcad-13961f6d2c6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:01.881676 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c3884-af3b-4941-bcad-13961f6d2c6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:01.883434 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c3884-af3b-4941-bcad-13961f6d2c6f service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:01.896748 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1039/2099] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:36:01 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:36:02.049366 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7fcce5-9c03-4048-a5dd-36f7a877648c None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:02.051150 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2d7fcce5-9c03-4048-a5dd-36f7a877648c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:02.069331 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7fcce5-9c03-4048-a5dd-36f7a877648c None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:02.073294 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8376fec-324e-4fb7-a606-ee9752fd931f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:02.073973 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8376fec-324e-4fb7-a606-ee9752fd931f service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:02.074262 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8376fec-324e-4fb7-a606-ee9752fd931f service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:02.074674 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8376fec-324e-4fb7-a606-ee9752fd931f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:02.157937 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8376fec-324e-4fb7-a606-ee9752fd931f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:02.160825 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8376fec-324e-4fb7-a606-ee9752fd931f service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:02.175651 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1061/2100] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:36:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:36:04.614493 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:04.791525 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:05.426205 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:05.500061 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:09.127667 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c3884-af3b-4941-bcad-13961f6d2c6f service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:09.130340 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a41c3884-af3b-4941-bcad-13961f6d2c6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:09.414684 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c3884-af3b-4941-bcad-13961f6d2c6f service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:09.426623 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bded97-dc25-4105-96ff-449f7dd71f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:09.432369 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bded97-dc25-4105-96ff-449f7dd71f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:09.432369 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bded97-dc25-4105-96ff-449f7dd71f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:09.432369 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bded97-dc25-4105-96ff-449f7dd71f0c admin admin] PATH_INFO: `/v3/users/3bd60029480046e4a7730d0e1a1dd892` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:09.433463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bded97-dc25-4105-96ff-449f7dd71f0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bd60029480046e4a7730d0e1a1dd892)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:09.442922 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bded97-dc25-4105-96ff-449f7dd71f0c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:09.747710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-51bded97-dc25-4105-96ff-449f7dd71f0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bd60029480046e4a7730d0e1a1dd892'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:09.749743 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-51bded97-dc25-4105-96ff-449f7dd71f0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bd60029480046e4a7730d0e1a1dd892'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:09.846526 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-51bded97-dc25-4105-96ff-449f7dd71f0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bd60029480046e4a7730d0e1a1dd892'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:09.863190 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-51bded97-dc25-4105-96ff-449f7dd71f0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bd60029480046e4a7730d0e1a1dd892'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:09.866151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-51bded97-dc25-4105-96ff-449f7dd71f0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bd60029480046e4a7730d0e1a1dd892'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:09.913153 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1040/2101] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:36:09 2022] DELETE /identity/v3/users/3bd60029480046e4a7730d0e1a1dd892 => generated 0 bytes in 787 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:09.919699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33b72b-e534-48e7-9a8c-9c588732856e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:09.920636 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33b72b-e534-48e7-9a8c-9c588732856e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:09.921265 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33b72b-e534-48e7-9a8c-9c588732856e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:10.059404 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1c33b72b-e534-48e7-9a8c-9c588732856e None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:10.065781 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1c33b72b-e534-48e7-9a8c-9c588732856e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:10.212712 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1062/2102] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:10.235296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bded97-dc25-4105-96ff-449f7dd71f0c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:10.243547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-51bded97-dc25-4105-96ff-449f7dd71f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:10.405773 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bded97-dc25-4105-96ff-449f7dd71f0c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:10.410868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de680b4d-5157-4cfb-a3ee-33a49e46113a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:10.411627 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de680b4d-5157-4cfb-a3ee-33a49e46113a service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:10.411917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de680b4d-5157-4cfb-a3ee-33a49e46113a service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:10.412197 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de680b4d-5157-4cfb-a3ee-33a49e46113a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:10.427791 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de680b4d-5157-4cfb-a3ee-33a49e46113a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:10.433145 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de680b4d-5157-4cfb-a3ee-33a49e46113a service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:10.454450 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1041/2103] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:36:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:36:11.353878 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33b72b-e534-48e7-9a8c-9c588732856e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:11.361046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1c33b72b-e534-48e7-9a8c-9c588732856e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:11.370184 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33b72b-e534-48e7-9a8c-9c588732856e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:11.376119 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d850b-0984-499f-993c-47429ee8c5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:11.377387 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d850b-0984-499f-993c-47429ee8c5eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:11.377909 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d850b-0984-499f-993c-47429ee8c5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:11.378276 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d850b-0984-499f-993c-47429ee8c5eb admin admin] PATH_INFO: `/v3/projects/16cee814ae7e491b8c4c4f880f17e1ba` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:11.379907 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d850b-0984-499f-993c-47429ee8c5eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=16cee814ae7e491b8c4c4f880f17e1ba)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:11.382636 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2d850b-0984-499f-993c-47429ee8c5eb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:11.567706 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3d2d850b-0984-499f-993c-47429ee8c5eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16cee814ae7e491b8c4c4f880f17e1ba', 'request_id': 'req-3d2d850b-0984-499f-993c-47429ee8c5eb'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:11.670097 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3d2d850b-0984-499f-993c-47429ee8c5eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16cee814ae7e491b8c4c4f880f17e1ba', 'request_id': 'req-3d2d850b-0984-499f-993c-47429ee8c5eb'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:11.685072 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3d2d850b-0984-499f-993c-47429ee8c5eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16cee814ae7e491b8c4c4f880f17e1ba', 'request_id': 'req-3d2d850b-0984-499f-993c-47429ee8c5eb'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:11.700992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3d2d850b-0984-499f-993c-47429ee8c5eb admin admin] The token cache is being invalidate because project 16cee814ae7e491b8c4c4f880f17e1ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:36:11.701289 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3d2d850b-0984-499f-993c-47429ee8c5eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:11.701954 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3d2d850b-0984-499f-993c-47429ee8c5eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:11.704129 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1063/2104] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:36:11 2022] DELETE /identity/v3/projects/16cee814ae7e491b8c4c4f880f17e1ba => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:11.710889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00b3e6-eaf1-410b-98d6-cc110fa42d16 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:11.711427 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00b3e6-eaf1-410b-98d6-cc110fa42d16 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:11.711427 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00b3e6-eaf1-410b-98d6-cc110fa42d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:11.731404 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ee00b3e6-eaf1-410b-98d6-cc110fa42d16 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:11.732764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ee00b3e6-eaf1-410b-98d6-cc110fa42d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:11.741793 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:11.776091 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0353ca6-78b1-4c28-935c-d4c4b751bfd5 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:11.778265 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0353ca6-78b1-4c28-935c-d4c4b751bfd5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:11.779263 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0353ca6-78b1-4c28-935c-d4c4b751bfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:11.805379 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a0353ca6-78b1-4c28-935c-d4c4b751bfd5 None None] MFA Rules not processed for user `2de23770f4374decb95ff94f3e7f793b`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:11.807419 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a0353ca6-78b1-4c28-935c-d4c4b751bfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:11.812910 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1042/2105] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:11.826828 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da6ef1-a47d-486c-a0a2-4c653021742a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:11.827158 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da6ef1-a47d-486c-a0a2-4c653021742a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:11.827726 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21da6ef1-a47d-486c-a0a2-4c653021742a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:11.872805 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-21da6ef1-a47d-486c-a0a2-4c653021742a None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:11.874061 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-21da6ef1-a47d-486c-a0a2-4c653021742a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:11.887882 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1043/2106] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:36:11.912223 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da6ef1-a47d-486c-a0a2-4c653021742a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:11.930409 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1064/2107] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:36:11.930685 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21da6ef1-a47d-486c-a0a2-4c653021742a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:11.935697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640fedd-47c0-45c9-976c-b7642d7afbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:11.936760 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640fedd-47c0-45c9-976c-b7642d7afbe0 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:11.937529 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640fedd-47c0-45c9-976c-b7642d7afbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:11.937998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640fedd-47c0-45c9-976c-b7642d7afbe0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:11.939777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640fedd-47c0-45c9-976c-b7642d7afbe0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:36:11.940538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640fedd-47c0-45c9-976c-b7642d7afbe0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:11.942923 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640fedd-47c0-45c9-976c-b7642d7afbe0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:11.960247 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1044/2108] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:36:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:36:11.966060 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0353ca6-78b1-4c28-935c-d4c4b751bfd5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:11.979153 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0353ca6-78b1-4c28-935c-d4c4b751bfd5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:11.984392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c8e1d-4770-49d7-b8cb-7bbf357e9c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:11.985238 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c8e1d-4770-49d7-b8cb-7bbf357e9c8a admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:11.985623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c8e1d-4770-49d7-b8cb-7bbf357e9c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:11.986028 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c8e1d-4770-49d7-b8cb-7bbf357e9c8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:11.986853 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c8e1d-4770-49d7-b8cb-7bbf357e9c8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:11.989058 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c8e1d-4770-49d7-b8cb-7bbf357e9c8a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:11.993032 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:12.087770 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640fedd-47c0-45c9-976c-b7642d7afbe0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:12.090539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5640fedd-47c0-45c9-976c-b7642d7afbe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:12.113148 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640fedd-47c0-45c9-976c-b7642d7afbe0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:12.120485 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f42272-6625-4f53-ba31-ee9973504b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:12.122058 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f42272-6625-4f53-ba31-ee9973504b4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:12.122538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f42272-6625-4f53-ba31-ee9973504b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:12.122940 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f42272-6625-4f53-ba31-ee9973504b4b admin admin] PATH_INFO: `/v3/users/362a81f2165c4376a0cc4041e037253c` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:12.125031 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f42272-6625-4f53-ba31-ee9973504b4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=362a81f2165c4376a0cc4041e037253c)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:12.129204 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f42272-6625-4f53-ba31-ee9973504b4b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:12.146850 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1065/2109] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:36:11 2022] POST /identity/v3/projects => generated 371 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:36:12.151385 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c8e1d-4770-49d7-b8cb-7bbf357e9c8a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:12.259508 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c8e1d-4770-49d7-b8cb-7bbf357e9c8a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:12.268148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c262650-7ac2-4d0d-9551-ca71b38b7aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:12.269312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c262650-7ac2-4d0d-9551-ca71b38b7aa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:12.269893 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c262650-7ac2-4d0d-9551-ca71b38b7aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:12.270871 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c262650-7ac2-4d0d-9551-ca71b38b7aa7 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:12.272326 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c262650-7ac2-4d0d-9551-ca71b38b7aa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:12.276040 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c262650-7ac2-4d0d-9551-ca71b38b7aa7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:12.378126 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-78f42272-6625-4f53-ba31-ee9973504b4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '362a81f2165c4376a0cc4041e037253c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:12.380037 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-78f42272-6625-4f53-ba31-ee9973504b4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '362a81f2165c4376a0cc4041e037253c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:12.395536 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1066/2110] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:36:12 2022] POST /identity/v3/users => generated 286 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:36:12.398573 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c262650-7ac2-4d0d-9551-ca71b38b7aa7 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:12.415793 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c262650-7ac2-4d0d-9551-ca71b38b7aa7 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:12.424387 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b8eb9-16a7-46db-9a22-617f13d97cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:12.425616 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b8eb9-16a7-46db-9a22-617f13d97cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:12.426153 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b8eb9-16a7-46db-9a22-617f13d97cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:12.428754 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b8eb9-16a7-46db-9a22-617f13d97cb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:12.428754 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2b8eb9-16a7-46db-9a22-617f13d97cb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:12.431601 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2b8eb9-16a7-46db-9a22-617f13d97cb7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:12.454247 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1067/2111] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:36:12.461095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b8eb9-16a7-46db-9a22-617f13d97cb7 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:12.479665 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b8eb9-16a7-46db-9a22-617f13d97cb7 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:12.485949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73288557-e899-40d8-ada4-259e4dde85d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:12.487063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73288557-e899-40d8-ada4-259e4dde85d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:12.487390 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73288557-e899-40d8-ada4-259e4dde85d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:12.487708 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73288557-e899-40d8-ada4-259e4dde85d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:12.488688 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73288557-e899-40d8-ada4-259e4dde85d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:12.490513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73288557-e899-40d8-ada4-259e4dde85d1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:12.499105 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-78f42272-6625-4f53-ba31-ee9973504b4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '362a81f2165c4376a0cc4041e037253c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:12.504313 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1068/2112] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:36:12.510702 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-78f42272-6625-4f53-ba31-ee9973504b4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '362a81f2165c4376a0cc4041e037253c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:12.510927 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73288557-e899-40d8-ada4-259e4dde85d1 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:12.512731 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-78f42272-6625-4f53-ba31-ee9973504b4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '362a81f2165c4376a0cc4041e037253c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:12.516906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-73288557-e899-40d8-ada4-259e4dde85d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:12.543661 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73288557-e899-40d8-ada4-259e4dde85d1 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:12.550611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce206adf-f647-4083-a395-7a42c50fee2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:12.552581 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce206adf-f647-4083-a395-7a42c50fee2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:12.553686 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce206adf-f647-4083-a395-7a42c50fee2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:12.554147 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce206adf-f647-4083-a395-7a42c50fee2b admin admin] PATH_INFO: `/v3/projects/19b35fc495a345898167db62b9b95887/users/b1a5d200007a4474a2a25d2ae0915c14/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:12.556918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1045/2113] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:36:12 2022] DELETE /identity/v3/users/362a81f2165c4376a0cc4041e037253c => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:36:12.562982 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614b03f-e54b-4b2b-84d2-cd3cbeaf03c9 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:12.563577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614b03f-e54b-4b2b-84d2-cd3cbeaf03c9 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:12.563994 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614b03f-e54b-4b2b-84d2-cd3cbeaf03c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:12.577808 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce206adf-f647-4083-a395-7a42c50fee2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=19b35fc495a345898167db62b9b95887, user_id=b1a5d200007a4474a2a25d2ae0915c14, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:12.581578 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce206adf-f647-4083-a395-7a42c50fee2b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:12.589840 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2614b03f-e54b-4b2b-84d2-cd3cbeaf03c9 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:12.591337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2614b03f-e54b-4b2b-84d2-cd3cbeaf03c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:12.698292 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1046/2114] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:12.719312 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614b03f-e54b-4b2b-84d2-cd3cbeaf03c9 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:12.723327 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2614b03f-e54b-4b2b-84d2-cd3cbeaf03c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:12.793221 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1069/2115] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:36:12 2022] PUT /identity/v3/projects/19b35fc495a345898167db62b9b95887/users/b1a5d200007a4474a2a25d2ae0915c14/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:12.795768 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28cbe1-abad-439a-818b-4435c4b0cd7f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:12.796568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28cbe1-abad-439a-818b-4435c4b0cd7f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:12.797130 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad28cbe1-abad-439a-818b-4435c4b0cd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:12.822560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ad28cbe1-abad-439a-818b-4435c4b0cd7f None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:12.824067 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ad28cbe1-abad-439a-818b-4435c4b0cd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:12.827808 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614b03f-e54b-4b2b-84d2-cd3cbeaf03c9 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:12.895397 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1070/2116] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:36:12.901669 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81f8ec-1dcb-4546-92bb-baf0d2d34b32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:12.903291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81f8ec-1dcb-4546-92bb-baf0d2d34b32 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:12.903697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81f8ec-1dcb-4546-92bb-baf0d2d34b32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:12.904199 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81f8ec-1dcb-4546-92bb-baf0d2d34b32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:12.908177 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad28cbe1-abad-439a-818b-4435c4b0cd7f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:12.917202 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81f8ec-1dcb-4546-92bb-baf0d2d34b32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:12.922157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81f8ec-1dcb-4546-92bb-baf0d2d34b32 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:12.935550 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad28cbe1-abad-439a-818b-4435c4b0cd7f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:12.940107 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90e49b-1547-4008-86a1-e1c23cb241b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:12.941419 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90e49b-1547-4008-86a1-e1c23cb241b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:12.941780 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90e49b-1547-4008-86a1-e1c23cb241b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:12.942201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90e49b-1547-4008-86a1-e1c23cb241b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:12.947814 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1047/2117] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:36:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:36:12.958574 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90e49b-1547-4008-86a1-e1c23cb241b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:12.960407 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb90e49b-1547-4008-86a1-e1c23cb241b4 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:12.976797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1071/2118] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:36:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:36:14.058877 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81f8ec-1dcb-4546-92bb-baf0d2d34b32 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:14.058877 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8a81f8ec-1dcb-4546-92bb-baf0d2d34b32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:14.075934 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81f8ec-1dcb-4546-92bb-baf0d2d34b32 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:14.082039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b57147-7397-4be3-b5ea-deb4eb42b930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:14.083207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b57147-7397-4be3-b5ea-deb4eb42b930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:14.083737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b57147-7397-4be3-b5ea-deb4eb42b930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:14.084013 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b57147-7397-4be3-b5ea-deb4eb42b930 admin admin] PATH_INFO: `/v3/users/85234e3d7b0c4a4d9df4688c7143101b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:14.085458 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b57147-7397-4be3-b5ea-deb4eb42b930 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85234e3d7b0c4a4d9df4688c7143101b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:14.087685 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b57147-7397-4be3-b5ea-deb4eb42b930 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:14.126330 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90e49b-1547-4008-86a1-e1c23cb241b4 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:14.128722 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cb90e49b-1547-4008-86a1-e1c23cb241b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:14.143979 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb90e49b-1547-4008-86a1-e1c23cb241b4 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:14.154166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0e753-0a2d-4e68-92ae-7cfb3c57c9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:14.157677 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0e753-0a2d-4e68-92ae-7cfb3c57c9bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:14.158231 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0e753-0a2d-4e68-92ae-7cfb3c57c9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:14.158905 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0e753-0a2d-4e68-92ae-7cfb3c57c9bd admin admin] PATH_INFO: `/v3/projects/67e4bbb83c0b4f4585b0d0afd6e6a35e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:14.160639 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec0e753-0a2d-4e68-92ae-7cfb3c57c9bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=67e4bbb83c0b4f4585b0d0afd6e6a35e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:14.164635 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec0e753-0a2d-4e68-92ae-7cfb3c57c9bd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:14.366403 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3ec0e753-0a2d-4e68-92ae-7cfb3c57c9bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67e4bbb83c0b4f4585b0d0afd6e6a35e', 'request_id': 'req-3ec0e753-0a2d-4e68-92ae-7cfb3c57c9bd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:14.379984 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01b57147-7397-4be3-b5ea-deb4eb42b930 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85234e3d7b0c4a4d9df4688c7143101b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:14.382916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01b57147-7397-4be3-b5ea-deb4eb42b930 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85234e3d7b0c4a4d9df4688c7143101b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:14.447434 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3ec0e753-0a2d-4e68-92ae-7cfb3c57c9bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67e4bbb83c0b4f4585b0d0afd6e6a35e', 'request_id': 'req-3ec0e753-0a2d-4e68-92ae-7cfb3c57c9bd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:14.463223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3ec0e753-0a2d-4e68-92ae-7cfb3c57c9bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67e4bbb83c0b4f4585b0d0afd6e6a35e', 'request_id': 'req-3ec0e753-0a2d-4e68-92ae-7cfb3c57c9bd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:14.496226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01b57147-7397-4be3-b5ea-deb4eb42b930 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85234e3d7b0c4a4d9df4688c7143101b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:14.513465 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3ec0e753-0a2d-4e68-92ae-7cfb3c57c9bd admin admin] The token cache is being invalidate because project 67e4bbb83c0b4f4585b0d0afd6e6a35e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:36:14.513996 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3ec0e753-0a2d-4e68-92ae-7cfb3c57c9bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:14.515255 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3ec0e753-0a2d-4e68-92ae-7cfb3c57c9bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:14.519190 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1072/2119] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:36:14 2022] DELETE /identity/v3/projects/67e4bbb83c0b4f4585b0d0afd6e6a35e => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:36:14.526575 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0e753-0a2d-4e68-92ae-7cfb3c57c9bd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:14.528919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3ec0e753-0a2d-4e68-92ae-7cfb3c57c9bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:14.545506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01b57147-7397-4be3-b5ea-deb4eb42b930 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85234e3d7b0c4a4d9df4688c7143101b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:14.545506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-01b57147-7397-4be3-b5ea-deb4eb42b930 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85234e3d7b0c4a4d9df4688c7143101b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:14.574077 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1048/2120] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:36:14 2022] DELETE /identity/v3/users/85234e3d7b0c4a4d9df4688c7143101b => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:36:14.578892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f904a-1472-4bfc-a72a-2d998b62b9f7 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:14.579527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f904a-1472-4bfc-a72a-2d998b62b9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:14.580212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320f904a-1472-4bfc-a72a-2d998b62b9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:14.605101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-320f904a-1472-4bfc-a72a-2d998b62b9f7 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:14.605930 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-320f904a-1472-4bfc-a72a-2d998b62b9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:14.718604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0e753-0a2d-4e68-92ae-7cfb3c57c9bd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:14.720525 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1049/2121] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:36:14.727923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d206d9-ff54-4bd2-897d-a073e679910c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:14.729771 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d206d9-ff54-4bd2-897d-a073e679910c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:14.730284 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d206d9-ff54-4bd2-897d-a073e679910c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:14.730846 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d206d9-ff54-4bd2-897d-a073e679910c admin admin] PATH_INFO: `/v3/users/eb199dc2a4424567b469e56d3785aec0` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:14.733551 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d206d9-ff54-4bd2-897d-a073e679910c admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb199dc2a4424567b469e56d3785aec0)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:14.734433 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f904a-1472-4bfc-a72a-2d998b62b9f7 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:14.736786 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-320f904a-1472-4bfc-a72a-2d998b62b9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:14.737167 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d206d9-ff54-4bd2-897d-a073e679910c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:14.834645 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:14.836007 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320f904a-1472-4bfc-a72a-2d998b62b9f7 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:14.842166 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda9b19-5319-4977-b204-bb0e20e5aad8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:14.843011 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda9b19-5319-4977-b204-bb0e20e5aad8 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:14.843396 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda9b19-5319-4977-b204-bb0e20e5aad8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:14.843977 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda9b19-5319-4977-b204-bb0e20e5aad8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:14.856107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda9b19-5319-4977-b204-bb0e20e5aad8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:14.857727 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda9b19-5319-4977-b204-bb0e20e5aad8 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:14.877347 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1050/2122] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:36:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:36:15.062981 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b9d206d9-ff54-4bd2-897d-a073e679910c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb199dc2a4424567b469e56d3785aec0'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:15.063722 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b9d206d9-ff54-4bd2-897d-a073e679910c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb199dc2a4424567b469e56d3785aec0'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:15.071968 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b9d206d9-ff54-4bd2-897d-a073e679910c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb199dc2a4424567b469e56d3785aec0'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:15.161210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b9d206d9-ff54-4bd2-897d-a073e679910c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb199dc2a4424567b469e56d3785aec0'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:15.162695 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b9d206d9-ff54-4bd2-897d-a073e679910c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb199dc2a4424567b469e56d3785aec0'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:15.219947 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1073/2123] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:36:14 2022] DELETE /identity/v3/users/eb199dc2a4424567b469e56d3785aec0 => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:36:15.304901 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:15.422466 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda9b19-5319-4977-b204-bb0e20e5aad8 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:15.425186 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8fda9b19-5319-4977-b204-bb0e20e5aad8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:15.454192 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda9b19-5319-4977-b204-bb0e20e5aad8 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:15.556574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db89c23-2380-402e-a774-5758a0290f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:15.557772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db89c23-2380-402e-a774-5758a0290f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:15.558200 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db89c23-2380-402e-a774-5758a0290f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:15.558810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db89c23-2380-402e-a774-5758a0290f3e admin admin] PATH_INFO: `/v3/projects/700cea58747346449902b9cd60948e5b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:15.560402 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db89c23-2380-402e-a774-5758a0290f3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=700cea58747346449902b9cd60948e5b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:15.562996 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db89c23-2380-402e-a774-5758a0290f3e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:15.721789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7db89c23-2380-402e-a774-5758a0290f3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '700cea58747346449902b9cd60948e5b', 'request_id': 'req-7db89c23-2380-402e-a774-5758a0290f3e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:15.742173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7db89c23-2380-402e-a774-5758a0290f3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '700cea58747346449902b9cd60948e5b', 'request_id': 'req-7db89c23-2380-402e-a774-5758a0290f3e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:15.750409 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7db89c23-2380-402e-a774-5758a0290f3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '700cea58747346449902b9cd60948e5b', 'request_id': 'req-7db89c23-2380-402e-a774-5758a0290f3e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:15.840100 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7db89c23-2380-402e-a774-5758a0290f3e admin admin] The token cache is being invalidate because project 700cea58747346449902b9cd60948e5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:36:15.841296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7db89c23-2380-402e-a774-5758a0290f3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:15.843273 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7db89c23-2380-402e-a774-5758a0290f3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:15.847553 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1051/2124] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:36:15 2022] DELETE /identity/v3/projects/700cea58747346449902b9cd60948e5b => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:15.850173 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d206d9-ff54-4bd2-897d-a073e679910c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:15.851661 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b9d206d9-ff54-4bd2-897d-a073e679910c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:15.869345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d206d9-ff54-4bd2-897d-a073e679910c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:15.958277 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f1da2-81df-4316-938d-c5fac018c171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:15.960164 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f1da2-81df-4316-938d-c5fac018c171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:15.960647 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f1da2-81df-4316-938d-c5fac018c171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:15.961091 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f1da2-81df-4316-938d-c5fac018c171 admin admin] PATH_INFO: `/v3/users/f96ede345afc4e2082eea8ae130990f6` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:15.962727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f1da2-81df-4316-938d-c5fac018c171 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f96ede345afc4e2082eea8ae130990f6)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:15.965312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f1da2-81df-4316-938d-c5fac018c171 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:16.034868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db89c23-2380-402e-a774-5758a0290f3e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:16.036446 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7db89c23-2380-402e-a774-5758a0290f3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:16.057661 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db89c23-2380-402e-a774-5758a0290f3e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:16.080494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad763c2-8889-4651-a6f6-cfa9e028767c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:16.081649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad763c2-8889-4651-a6f6-cfa9e028767c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:16.082004 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad763c2-8889-4651-a6f6-cfa9e028767c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:16.082298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad763c2-8889-4651-a6f6-cfa9e028767c admin admin] PATH_INFO: `/v3/projects/4b8b725a6ef4466fbe50bf3bf7073dc2` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:16.084149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad763c2-8889-4651-a6f6-cfa9e028767c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b8b725a6ef4466fbe50bf3bf7073dc2)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:16.087761 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad763c2-8889-4651-a6f6-cfa9e028767c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:16.234320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-145f1da2-81df-4316-938d-c5fac018c171 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f96ede345afc4e2082eea8ae130990f6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:16.239926 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-145f1da2-81df-4316-938d-c5fac018c171 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f96ede345afc4e2082eea8ae130990f6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:16.271843 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-145f1da2-81df-4316-938d-c5fac018c171 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f96ede345afc4e2082eea8ae130990f6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:16.317331 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0ad763c2-8889-4651-a6f6-cfa9e028767c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b8b725a6ef4466fbe50bf3bf7073dc2', 'request_id': 'req-0ad763c2-8889-4651-a6f6-cfa9e028767c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:16.326234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0ad763c2-8889-4651-a6f6-cfa9e028767c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b8b725a6ef4466fbe50bf3bf7073dc2', 'request_id': 'req-0ad763c2-8889-4651-a6f6-cfa9e028767c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:16.341196 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0ad763c2-8889-4651-a6f6-cfa9e028767c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b8b725a6ef4466fbe50bf3bf7073dc2', 'request_id': 'req-0ad763c2-8889-4651-a6f6-cfa9e028767c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:16.355692 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-145f1da2-81df-4316-938d-c5fac018c171 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f96ede345afc4e2082eea8ae130990f6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:16.356313 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-145f1da2-81df-4316-938d-c5fac018c171 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f96ede345afc4e2082eea8ae130990f6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:16.378593 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1074/2125] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:36:15 2022] DELETE /identity/v3/users/f96ede345afc4e2082eea8ae130990f6 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:16.511878 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0ad763c2-8889-4651-a6f6-cfa9e028767c admin admin] The token cache is being invalidate because project 4b8b725a6ef4466fbe50bf3bf7073dc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:36:16.512434 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0ad763c2-8889-4651-a6f6-cfa9e028767c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:16.512927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0ad763c2-8889-4651-a6f6-cfa9e028767c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:16.514583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1052/2126] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:36:16 2022] DELETE /identity/v3/projects/4b8b725a6ef4466fbe50bf3bf7073dc2 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:16.519396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f1da2-81df-4316-938d-c5fac018c171 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:16.521056 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-145f1da2-81df-4316-938d-c5fac018c171 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:16.605430 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f1da2-81df-4316-938d-c5fac018c171 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:16.610679 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f1094-4f7e-4098-8c57-4e61aaf871c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:16.611824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f1094-4f7e-4098-8c57-4e61aaf871c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:16.612302 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f1094-4f7e-4098-8c57-4e61aaf871c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:16.612803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f1094-4f7e-4098-8c57-4e61aaf871c8 admin admin] PATH_INFO: `/v3/users/f61d0c1c0ddc45849468e72ce734df4c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:16.614023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255f1094-4f7e-4098-8c57-4e61aaf871c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f61d0c1c0ddc45849468e72ce734df4c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:16.616094 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255f1094-4f7e-4098-8c57-4e61aaf871c8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:16.963306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-255f1094-4f7e-4098-8c57-4e61aaf871c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f61d0c1c0ddc45849468e72ce734df4c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:16.964080 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-255f1094-4f7e-4098-8c57-4e61aaf871c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f61d0c1c0ddc45849468e72ce734df4c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:16.981922 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-255f1094-4f7e-4098-8c57-4e61aaf871c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f61d0c1c0ddc45849468e72ce734df4c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:17.226510 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-255f1094-4f7e-4098-8c57-4e61aaf871c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f61d0c1c0ddc45849468e72ce734df4c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:17.227441 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-255f1094-4f7e-4098-8c57-4e61aaf871c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f61d0c1c0ddc45849468e72ce734df4c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:17.249575 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1075/2127] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:36:16 2022] DELETE /identity/v3/users/f61d0c1c0ddc45849468e72ce734df4c => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:36:17.315450 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad763c2-8889-4651-a6f6-cfa9e028767c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:17.318166 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0ad763c2-8889-4651-a6f6-cfa9e028767c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:17.612561 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad763c2-8889-4651-a6f6-cfa9e028767c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:17.678846 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:17.750794 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976f4ea-529a-4ef9-bb97-2448a53ceab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:17.753046 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976f4ea-529a-4ef9-bb97-2448a53ceab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:17.753603 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976f4ea-529a-4ef9-bb97-2448a53ceab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:17.754101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976f4ea-529a-4ef9-bb97-2448a53ceab0 admin admin] PATH_INFO: `/v3/projects/c6c3da678b484b079729c603fcbb6f6b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:17.760583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976f4ea-529a-4ef9-bb97-2448a53ceab0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6c3da678b484b079729c603fcbb6f6b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:17.763260 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976f4ea-529a-4ef9-bb97-2448a53ceab0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:17.981260 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d976f4ea-529a-4ef9-bb97-2448a53ceab0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6c3da678b484b079729c603fcbb6f6b', 'request_id': 'req-d976f4ea-529a-4ef9-bb97-2448a53ceab0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:18.013876 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d976f4ea-529a-4ef9-bb97-2448a53ceab0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6c3da678b484b079729c603fcbb6f6b', 'request_id': 'req-d976f4ea-529a-4ef9-bb97-2448a53ceab0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:18.040563 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d976f4ea-529a-4ef9-bb97-2448a53ceab0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6c3da678b484b079729c603fcbb6f6b', 'request_id': 'req-d976f4ea-529a-4ef9-bb97-2448a53ceab0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:18.195594 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d976f4ea-529a-4ef9-bb97-2448a53ceab0 admin admin] The token cache is being invalidate because project c6c3da678b484b079729c603fcbb6f6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:36:18.195594 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d976f4ea-529a-4ef9-bb97-2448a53ceab0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:18.195594 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d976f4ea-529a-4ef9-bb97-2448a53ceab0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:18.197976 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1053/2128] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:36:17 2022] DELETE /identity/v3/projects/c6c3da678b484b079729c603fcbb6f6b => generated 0 bytes in 883 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:18.202994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0054d8-2d11-4748-9ff6-3eafdf03cd36 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:18.203885 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0054d8-2d11-4748-9ff6-3eafdf03cd36 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:18.204322 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0054d8-2d11-4748-9ff6-3eafdf03cd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:18.229627 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ec0054d8-2d11-4748-9ff6-3eafdf03cd36 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:18.231109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ec0054d8-2d11-4748-9ff6-3eafdf03cd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:18.329178 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1076/2129] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:18.337743 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295c385-db98-4217-8301-06912be37b5e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:18.338200 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295c385-db98-4217-8301-06912be37b5e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:18.338576 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295c385-db98-4217-8301-06912be37b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:18.370704 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f295c385-db98-4217-8301-06912be37b5e None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:18.372118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f295c385-db98-4217-8301-06912be37b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:18.385699 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1054/2130] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:18.387068 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0054d8-2d11-4748-9ff6-3eafdf03cd36 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:18.402927 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0054d8-2d11-4748-9ff6-3eafdf03cd36 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:18.412592 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7463a-9002-44fe-ba32-63b12e80bfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:18.414801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7463a-9002-44fe-ba32-63b12e80bfa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:18.415428 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7463a-9002-44fe-ba32-63b12e80bfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:18.415925 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7463a-9002-44fe-ba32-63b12e80bfa0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:18.417094 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7463a-9002-44fe-ba32-63b12e80bfa0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:36:18.417578 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7463a-9002-44fe-ba32-63b12e80bfa0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:18.419779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7463a-9002-44fe-ba32-63b12e80bfa0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:18.446615 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1077/2131] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:36:18.453034 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295c385-db98-4217-8301-06912be37b5e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:18.470668 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295c385-db98-4217-8301-06912be37b5e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:18.482303 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e95438-d199-41af-bd36-40c08808a3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:18.483932 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e95438-d199-41af-bd36-40c08808a3e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:18.484436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e95438-d199-41af-bd36-40c08808a3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:18.484985 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e95438-d199-41af-bd36-40c08808a3e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:18.486042 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e95438-d199-41af-bd36-40c08808a3e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:18.492607 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e95438-d199-41af-bd36-40c08808a3e3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:18.632426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:18.641793 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1055/2132] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:36:18 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:36:18.648230 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7463a-9002-44fe-ba32-63b12e80bfa0 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:18.670893 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7463a-9002-44fe-ba32-63b12e80bfa0 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:18.696095 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e95438-d199-41af-bd36-40c08808a3e3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:18.699008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c3e95438-d199-41af-bd36-40c08808a3e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:18.837648 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76892592-a24a-4a6a-9eb4-b966bfa68ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:18.838878 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76892592-a24a-4a6a-9eb4-b966bfa68ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:18.849243 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76892592-a24a-4a6a-9eb4-b966bfa68ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:18.849243 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76892592-a24a-4a6a-9eb4-b966bfa68ae5 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:18.849243 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76892592-a24a-4a6a-9eb4-b966bfa68ae5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:18.849627 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e95438-d199-41af-bd36-40c08808a3e3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:18.849720 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76892592-a24a-4a6a-9eb4-b966bfa68ae5 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:18.858359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe40f8e-ff82-4094-aa0a-53150f2afe10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:18.859509 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe40f8e-ff82-4094-aa0a-53150f2afe10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:18.859939 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe40f8e-ff82-4094-aa0a-53150f2afe10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:18.860381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe40f8e-ff82-4094-aa0a-53150f2afe10 admin admin] PATH_INFO: `/v3/projects/e7ab1b5c8ad64c108c41149d7b33cd5a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:18.862529 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe40f8e-ff82-4094-aa0a-53150f2afe10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7ab1b5c8ad64c108c41149d7b33cd5a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:18.866325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe40f8e-ff82-4094-aa0a-53150f2afe10 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:19.031790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1078/2133] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:36:18 2022] POST /identity/v3/users => generated 292 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:36:19.033706 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76892592-a24a-4a6a-9eb4-b966bfa68ae5 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:19.059766 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76892592-a24a-4a6a-9eb4-b966bfa68ae5 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:19.083681 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3fe40f8e-ff82-4094-aa0a-53150f2afe10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7ab1b5c8ad64c108c41149d7b33cd5a', 'request_id': 'req-3fe40f8e-ff82-4094-aa0a-53150f2afe10'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:19.113669 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3fe40f8e-ff82-4094-aa0a-53150f2afe10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7ab1b5c8ad64c108c41149d7b33cd5a', 'request_id': 'req-3fe40f8e-ff82-4094-aa0a-53150f2afe10'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:19.132001 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3fe40f8e-ff82-4094-aa0a-53150f2afe10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7ab1b5c8ad64c108c41149d7b33cd5a', 'request_id': 'req-3fe40f8e-ff82-4094-aa0a-53150f2afe10'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:19.174699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd4d59-5653-41db-9af0-14f07aedecdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:19.176020 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd4d59-5653-41db-9af0-14f07aedecdc admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:19.176462 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd4d59-5653-41db-9af0-14f07aedecdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:19.176788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd4d59-5653-41db-9af0-14f07aedecdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:19.177722 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd4d59-5653-41db-9af0-14f07aedecdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:19.179605 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd4d59-5653-41db-9af0-14f07aedecdc admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:19.238026 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1079/2134] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:36:19.244015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd4d59-5653-41db-9af0-14f07aedecdc admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:19.244189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3fe40f8e-ff82-4094-aa0a-53150f2afe10 admin admin] The token cache is being invalidate because project e7ab1b5c8ad64c108c41149d7b33cd5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:36:19.244540 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3fe40f8e-ff82-4094-aa0a-53150f2afe10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:19.245327 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3fe40f8e-ff82-4094-aa0a-53150f2afe10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:19.247932 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1056/2135] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:36:18 2022] DELETE /identity/v3/projects/e7ab1b5c8ad64c108c41149d7b33cd5a => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:19.255129 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919b32f-d92e-4aa9-bbff-149a86c7500b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:19.255768 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919b32f-d92e-4aa9-bbff-149a86c7500b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:19.256250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919b32f-d92e-4aa9-bbff-149a86c7500b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:19.267740 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd4d59-5653-41db-9af0-14f07aedecdc admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:19.276238 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23188f-2008-493c-814a-04780a79b7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:19.277193 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23188f-2008-493c-814a-04780a79b7f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:19.278325 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23188f-2008-493c-814a-04780a79b7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:19.278325 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23188f-2008-493c-814a-04780a79b7f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:19.278511 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b919b32f-d92e-4aa9-bbff-149a86c7500b None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:19.279118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23188f-2008-493c-814a-04780a79b7f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:19.279563 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b919b32f-d92e-4aa9-bbff-149a86c7500b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:19.287749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23188f-2008-493c-814a-04780a79b7f2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:19.300708 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1080/2136] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:36:19.315940 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23188f-2008-493c-814a-04780a79b7f2 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:19.320135 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ea23188f-2008-493c-814a-04780a79b7f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:19.322811 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1057/2137] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:36:19.333094 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3c518-5d2a-4a96-9656-274dbfa738c5 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:19.333399 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3c518-5d2a-4a96-9656-274dbfa738c5 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:19.333652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3c518-5d2a-4a96-9656-274dbfa738c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:19.341351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23188f-2008-493c-814a-04780a79b7f2 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:19.350482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b84ea0-e3dd-40b2-88e0-ea874f12f1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:19.351679 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b84ea0-e3dd-40b2-88e0-ea874f12f1b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:19.351975 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b84ea0-e3dd-40b2-88e0-ea874f12f1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:19.352260 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b84ea0-e3dd-40b2-88e0-ea874f12f1b4 admin admin] PATH_INFO: `/v3/projects/6d289d6f2a0d44e4b070c5ea6bbec0d3/users/78d83b62d35a4b73a8bedcba9b13b93f/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:19.382896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b84ea0-e3dd-40b2-88e0-ea874f12f1b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d289d6f2a0d44e4b070c5ea6bbec0d3, user_id=78d83b62d35a4b73a8bedcba9b13b93f, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:19.385907 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b84ea0-e3dd-40b2-88e0-ea874f12f1b4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:19.399199 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-62a3c518-5d2a-4a96-9656-274dbfa738c5 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:19.399199 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-62a3c518-5d2a-4a96-9656-274dbfa738c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:19.409547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1058/2138] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:36:19.410706 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3c518-5d2a-4a96-9656-274dbfa738c5 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:19.421722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3c518-5d2a-4a96-9656-274dbfa738c5 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:19.427737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b0ad9-77f2-4e65-91c3-1090e8193afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:19.436599 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b0ad9-77f2-4e65-91c3-1090e8193afe admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:19.436599 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b0ad9-77f2-4e65-91c3-1090e8193afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:19.436599 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b0ad9-77f2-4e65-91c3-1090e8193afe admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:19.436599 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b0ad9-77f2-4e65-91c3-1090e8193afe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:36:19.436599 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b0ad9-77f2-4e65-91c3-1090e8193afe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:19.438802 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b0ad9-77f2-4e65-91c3-1090e8193afe admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:19.462891 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1059/2139] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:36:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:36:19.467202 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b0ad9-77f2-4e65-91c3-1090e8193afe admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:19.483196 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b0ad9-77f2-4e65-91c3-1090e8193afe admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:19.507204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1081/2140] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:36:19 2022] PUT /identity/v3/projects/6d289d6f2a0d44e4b070c5ea6bbec0d3/users/78d83b62d35a4b73a8bedcba9b13b93f/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:19.519426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eed51b-9157-4b5d-b85d-466987fe44d8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:19.519748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eed51b-9157-4b5d-b85d-466987fe44d8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:19.520266 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eed51b-9157-4b5d-b85d-466987fe44d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:19.569767 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-62eed51b-9157-4b5d-b85d-466987fe44d8 None None] MFA Rules not processed for user `78d83b62d35a4b73a8bedcba9b13b93f`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:19.571405 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-62eed51b-9157-4b5d-b85d-466987fe44d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:19.658618 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea75ff-8e0d-440b-8a45-13dbe744fc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:19.660415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea75ff-8e0d-440b-8a45-13dbe744fc56 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:19.660415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea75ff-8e0d-440b-8a45-13dbe744fc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:19.660415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea75ff-8e0d-440b-8a45-13dbe744fc56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:19.661107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea75ff-8e0d-440b-8a45-13dbe744fc56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:19.672007 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea75ff-8e0d-440b-8a45-13dbe744fc56 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:19.759256 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1082/2141] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:19.764446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eed51b-9157-4b5d-b85d-466987fe44d8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:19.787133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eed51b-9157-4b5d-b85d-466987fe44d8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:19.799522 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c3548-9fa4-4463-bb6a-4439ba95d138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:19.800416 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c3548-9fa4-4463-bb6a-4439ba95d138 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:19.800893 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c3548-9fa4-4463-bb6a-4439ba95d138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:19.801325 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c3548-9fa4-4463-bb6a-4439ba95d138 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:19.802314 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c3548-9fa4-4463-bb6a-4439ba95d138 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:19.804458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126c3548-9fa4-4463-bb6a-4439ba95d138 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:19.849125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1060/2142] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:36:19 2022] POST /identity/v3/projects => generated 385 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:36:19.855801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea75ff-8e0d-440b-8a45-13dbe744fc56 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:19.969339 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea75ff-8e0d-440b-8a45-13dbe744fc56 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:19.977245 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342cb74d-d66e-43e8-b3ca-fca95947fdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:19.978096 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cb74d-d66e-43e8-b3ca-fca95947fdb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:19.978767 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cb74d-d66e-43e8-b3ca-fca95947fdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:19.979054 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cb74d-d66e-43e8-b3ca-fca95947fdb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:19.979989 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342cb74d-d66e-43e8-b3ca-fca95947fdb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:19.981751 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1083/2143] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:36:19 2022] POST /identity/v3/projects => generated 383 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:36:19.982601 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342cb74d-d66e-43e8-b3ca-fca95947fdb4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:19.986314 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c3548-9fa4-4463-bb6a-4439ba95d138 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:20.168081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1061/2144] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:36:19 2022] POST /identity/v3/users => generated 293 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:36:20.169566 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342cb74d-d66e-43e8-b3ca-fca95947fdb4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:20.185324 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342cb74d-d66e-43e8-b3ca-fca95947fdb4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:20.210769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c3548-9fa4-4463-bb6a-4439ba95d138 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:20.213568 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312dc5b-b05d-4b64-82a7-bbff21c26df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:20.215427 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312dc5b-b05d-4b64-82a7-bbff21c26df3 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:20.215960 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312dc5b-b05d-4b64-82a7-bbff21c26df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:20.216389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3312dc5b-b05d-4b64-82a7-bbff21c26df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:20.219642 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3312dc5b-b05d-4b64-82a7-bbff21c26df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:20.225825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3312dc5b-b05d-4b64-82a7-bbff21c26df3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:20.228663 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95988f9-35b1-4c94-b641-07ae552bd32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:20.229416 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95988f9-35b1-4c94-b641-07ae552bd32e admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:20.229696 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95988f9-35b1-4c94-b641-07ae552bd32e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:20.230138 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95988f9-35b1-4c94-b641-07ae552bd32e admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:20.231203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95988f9-35b1-4c94-b641-07ae552bd32e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:20.233710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95988f9-35b1-4c94-b641-07ae552bd32e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:20.244915 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1062/2145] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:36:20.262039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312dc5b-b05d-4b64-82a7-bbff21c26df3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:20.283554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3312dc5b-b05d-4b64-82a7-bbff21c26df3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:20.290203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b4dab-5039-475e-8a6a-1f9465dc8467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:20.291030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b4dab-5039-475e-8a6a-1f9465dc8467 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:20.291317 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b4dab-5039-475e-8a6a-1f9465dc8467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:20.291640 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b4dab-5039-475e-8a6a-1f9465dc8467 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:20.292425 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b4dab-5039-475e-8a6a-1f9465dc8467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:20.294273 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b4dab-5039-475e-8a6a-1f9465dc8467 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:20.311518 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1063/2146] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:36:20.316727 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b4dab-5039-475e-8a6a-1f9465dc8467 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:20.328508 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b4dab-5039-475e-8a6a-1f9465dc8467 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:20.347529 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1084/2147] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:36:19 2022] POST /identity/v3/users => generated 298 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:36:20.350154 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5faeb-e3f5-4696-8473-29a76766e670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:20.351179 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5faeb-e3f5-4696-8473-29a76766e670 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:20.351458 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5faeb-e3f5-4696-8473-29a76766e670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:20.351793 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5faeb-e3f5-4696-8473-29a76766e670 admin admin] PATH_INFO: `/v3/projects/0908533e5a3345ecaeac47b5bd3526e8/users/d3a0c7877b0841b2b253b1ca01ba69c9/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:20.355255 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95988f9-35b1-4c94-b641-07ae552bd32e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:20.387407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d5faeb-e3f5-4696-8473-29a76766e670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0908533e5a3345ecaeac47b5bd3526e8, user_id=d3a0c7877b0841b2b253b1ca01ba69c9, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:20.391092 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95988f9-35b1-4c94-b641-07ae552bd32e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:20.391784 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d5faeb-e3f5-4696-8473-29a76766e670 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:20.406166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06001754-7e17-4ed3-9531-e7fbb77e9c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:20.408109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06001754-7e17-4ed3-9531-e7fbb77e9c97 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:20.409435 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06001754-7e17-4ed3-9531-e7fbb77e9c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:20.411517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06001754-7e17-4ed3-9531-e7fbb77e9c97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:20.412717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06001754-7e17-4ed3-9531-e7fbb77e9c97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:20.419434 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:20.419434 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06001754-7e17-4ed3-9531-e7fbb77e9c97 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:20.437744 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1085/2148] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:36:20.443931 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06001754-7e17-4ed3-9531-e7fbb77e9c97 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:20.465980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06001754-7e17-4ed3-9531-e7fbb77e9c97 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:20.483732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36b0bd-58fa-4c08-a36d-4632e776c08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:20.484818 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36b0bd-58fa-4c08-a36d-4632e776c08a admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:20.485106 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36b0bd-58fa-4c08-a36d-4632e776c08a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:20.485484 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36b0bd-58fa-4c08-a36d-4632e776c08a admin admin] PATH_INFO: `/v3/projects/986b61ce70d74220a5e945a34e3e1de0/users/96d2252a94a6431995f5d85abc65a491/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:20.530995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36b0bd-58fa-4c08-a36d-4632e776c08a admin admin] RBAC: Authorizing `identity:create_grant(project_id=986b61ce70d74220a5e945a34e3e1de0, user_id=96d2252a94a6431995f5d85abc65a491, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:20.544229 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36b0bd-58fa-4c08-a36d-4632e776c08a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:20.567528 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1064/2149] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:36:20 2022] PUT /identity/v3/projects/0908533e5a3345ecaeac47b5bd3526e8/users/d3a0c7877b0841b2b253b1ca01ba69c9/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:20.577241 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77f7e7-7157-49a2-9e5e-32fdad898378 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:20.577651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77f7e7-7157-49a2-9e5e-32fdad898378 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:20.578019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77f7e7-7157-49a2-9e5e-32fdad898378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:20.648395 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ca77f7e7-7157-49a2-9e5e-32fdad898378 None None] MFA Rules not processed for user `d3a0c7877b0841b2b253b1ca01ba69c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:20.650733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ca77f7e7-7157-49a2-9e5e-32fdad898378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:20.759423 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1086/2150] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:36:20 2022] PUT /identity/v3/projects/986b61ce70d74220a5e945a34e3e1de0/users/96d2252a94a6431995f5d85abc65a491/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:20.763048 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36b0bd-58fa-4c08-a36d-4632e776c08a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:20.837601 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1065/2151] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:36:20.843536 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77f7e7-7157-49a2-9e5e-32fdad898378 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:20.945734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36b0bd-58fa-4c08-a36d-4632e776c08a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:20.974345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab4148-711f-4400-b9f3-1e160cc64371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:20.975237 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab4148-711f-4400-b9f3-1e160cc64371 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:20.975595 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab4148-711f-4400-b9f3-1e160cc64371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:20.975918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab4148-711f-4400-b9f3-1e160cc64371 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:20.977195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab4148-711f-4400-b9f3-1e160cc64371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:20.979869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab4148-711f-4400-b9f3-1e160cc64371 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:21.009792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77f7e7-7157-49a2-9e5e-32fdad898378 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:21.014184 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1087/2152] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:20 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:36:21.019051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab4148-711f-4400-b9f3-1e160cc64371 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:21.028240 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b736f3f-1a32-46ea-8e25-5a08699d3438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:21.028938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b736f3f-1a32-46ea-8e25-5a08699d3438 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:21.029221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b736f3f-1a32-46ea-8e25-5a08699d3438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:21.029496 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b736f3f-1a32-46ea-8e25-5a08699d3438 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:21.030212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b736f3f-1a32-46ea-8e25-5a08699d3438 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:21.041169 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b736f3f-1a32-46ea-8e25-5a08699d3438 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:21.050855 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab4148-711f-4400-b9f3-1e160cc64371 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:21.061066 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373aeb29-2468-4770-b3ef-1a5663db03be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:21.062292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373aeb29-2468-4770-b3ef-1a5663db03be admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:21.062749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373aeb29-2468-4770-b3ef-1a5663db03be admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:21.063193 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373aeb29-2468-4770-b3ef-1a5663db03be admin admin] PATH_INFO: `/v3/projects/986b61ce70d74220a5e945a34e3e1de0/users/96d2252a94a6431995f5d85abc65a491/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:21.074472 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373aeb29-2468-4770-b3ef-1a5663db03be admin admin] RBAC: Authorizing `identity:create_grant(project_id=986b61ce70d74220a5e945a34e3e1de0, user_id=96d2252a94a6431995f5d85abc65a491, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:21.079024 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373aeb29-2468-4770-b3ef-1a5663db03be admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:21.091053 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:21.101596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1088/2153] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:36:21 2022] PUT /identity/v3/projects/986b61ce70d74220a5e945a34e3e1de0/users/96d2252a94a6431995f5d85abc65a491/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:21.112463 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e61ea-c913-4ef3-86db-6830f1ca8e8f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:21.112750 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e61ea-c913-4ef3-86db-6830f1ca8e8f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:21.113555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e61ea-c913-4ef3-86db-6830f1ca8e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:21.162255 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-293e61ea-c913-4ef3-86db-6830f1ca8e8f None None] MFA Rules not processed for user `96d2252a94a6431995f5d85abc65a491`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:21.172471 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-293e61ea-c913-4ef3-86db-6830f1ca8e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:21.209428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1066/2154] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:36:20 2022] POST /identity/v3/projects => generated 387 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:36:21.213179 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b736f3f-1a32-46ea-8e25-5a08699d3438 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:21.430222 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b736f3f-1a32-46ea-8e25-5a08699d3438 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:21.437365 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0662045-6870-4714-89ae-12a19fd0b187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:21.438912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0662045-6870-4714-89ae-12a19fd0b187 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:21.440944 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0662045-6870-4714-89ae-12a19fd0b187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:21.441625 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0662045-6870-4714-89ae-12a19fd0b187 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:21.444025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1089/2155] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:36:21.445138 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0662045-6870-4714-89ae-12a19fd0b187 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:21.461091 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0662045-6870-4714-89ae-12a19fd0b187 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:21.463627 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e61ea-c913-4ef3-86db-6830f1ca8e8f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:21.478255 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-293e61ea-c913-4ef3-86db-6830f1ca8e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:21.643234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1067/2156] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:36:21 2022] POST /identity/v3/users => generated 300 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:36:21.649366 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0662045-6870-4714-89ae-12a19fd0b187 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:21.674224 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0662045-6870-4714-89ae-12a19fd0b187 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:21.686237 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb238c-2d7a-4186-ab23-abb837745746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:21.687200 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb238c-2d7a-4186-ab23-abb837745746 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:21.687622 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb238c-2d7a-4186-ab23-abb837745746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:21.688008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb238c-2d7a-4186-ab23-abb837745746 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:21.688952 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eb238c-2d7a-4186-ab23-abb837745746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:21.691453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eb238c-2d7a-4186-ab23-abb837745746 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:21.727143 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1068/2157] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:36:21.731149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb238c-2d7a-4186-ab23-abb837745746 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:21.739753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e61ea-c913-4ef3-86db-6830f1ca8e8f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:21.744203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb238c-2d7a-4186-ab23-abb837745746 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:21.759769 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d91bff-f837-41c2-99e9-30cea2ff8291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:21.764067 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d91bff-f837-41c2-99e9-30cea2ff8291 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:21.764067 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d91bff-f837-41c2-99e9-30cea2ff8291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:21.764067 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d91bff-f837-41c2-99e9-30cea2ff8291 admin admin] PATH_INFO: `/v3/projects/e3ed410ca3a848baad97986417086b07/users/99133f76012742b2ae1765b2f4662f94/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:21.799378 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d91bff-f837-41c2-99e9-30cea2ff8291 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3ed410ca3a848baad97986417086b07, user_id=99133f76012742b2ae1765b2f4662f94, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:21.804946 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d91bff-f837-41c2-99e9-30cea2ff8291 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:21.862003 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26968cdf-7e94-4e3e-8e84-800d39098f3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:21.862875 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26968cdf-7e94-4e3e-8e84-800d39098f3b service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:21.863353 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26968cdf-7e94-4e3e-8e84-800d39098f3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:21.863671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26968cdf-7e94-4e3e-8e84-800d39098f3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:21.945273 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1069/2158] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:36:21 2022] PUT /identity/v3/projects/e3ed410ca3a848baad97986417086b07/users/99133f76012742b2ae1765b2f4662f94/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:36:21.949393 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d91bff-f837-41c2-99e9-30cea2ff8291 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:22.053783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26968cdf-7e94-4e3e-8e84-800d39098f3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:22.053783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26968cdf-7e94-4e3e-8e84-800d39098f3b service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:22.094991 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1090/2159] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:36:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 635 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:36:22.168379 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d91bff-f837-41c2-99e9-30cea2ff8291 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:22.174094 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffab210-59b4-4c74-8e57-3d099a2de174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:22.175378 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffab210-59b4-4c74-8e57-3d099a2de174 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:22.175885 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffab210-59b4-4c74-8e57-3d099a2de174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:22.176166 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffab210-59b4-4c74-8e57-3d099a2de174 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:22.177058 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffab210-59b4-4c74-8e57-3d099a2de174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:22.178886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffab210-59b4-4c74-8e57-3d099a2de174 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:22.201287 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1070/2160] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:21 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:36:22.206869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26968cdf-7e94-4e3e-8e84-800d39098f3b service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:22.226716 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26968cdf-7e94-4e3e-8e84-800d39098f3b service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:22.238095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6952d4c-c443-41e6-a88d-fb51fdb4d8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:22.239127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6952d4c-c443-41e6-a88d-fb51fdb4d8bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:22.239511 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6952d4c-c443-41e6-a88d-fb51fdb4d8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:22.239920 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6952d4c-c443-41e6-a88d-fb51fdb4d8bd admin admin] PATH_INFO: `/v3/projects/e3ed410ca3a848baad97986417086b07/users/99133f76012742b2ae1765b2f4662f94/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:22.242194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6952d4c-c443-41e6-a88d-fb51fdb4d8bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3ed410ca3a848baad97986417086b07, user_id=99133f76012742b2ae1765b2f4662f94, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:22.245433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6952d4c-c443-41e6-a88d-fb51fdb4d8bd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:22.260066 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1091/2161] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:36:22 2022] PUT /identity/v3/projects/e3ed410ca3a848baad97986417086b07/users/99133f76012742b2ae1765b2f4662f94/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:22.275065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17ece9-f31a-46db-b7d9-5f70ae7f5e3c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:22.275365 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17ece9-f31a-46db-b7d9-5f70ae7f5e3c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:22.275720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17ece9-f31a-46db-b7d9-5f70ae7f5e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:22.325075 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7f17ece9-f31a-46db-b7d9-5f70ae7f5e3c None None] MFA Rules not processed for user `99133f76012742b2ae1765b2f4662f94`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:22.326403 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7f17ece9-f31a-46db-b7d9-5f70ae7f5e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:22.373877 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6952d4c-c443-41e6-a88d-fb51fdb4d8bd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:22.376637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f6952d4c-c443-41e6-a88d-fb51fdb4d8bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:22.543164 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1071/2162] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:22 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:36:22.550203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6952d4c-c443-41e6-a88d-fb51fdb4d8bd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:22.561569 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eaaf72-2eb4-4b98-a6d6-4f7206cfa33c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:22.562177 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eaaf72-2eb4-4b98-a6d6-4f7206cfa33c service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:22.562465 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eaaf72-2eb4-4b98-a6d6-4f7206cfa33c service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:22.562745 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eaaf72-2eb4-4b98-a6d6-4f7206cfa33c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:22.580337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17ece9-f31a-46db-b7d9-5f70ae7f5e3c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:22.591413 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17ece9-f31a-46db-b7d9-5f70ae7f5e3c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:22.597851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff0c0c-677a-419e-bec0-7857d9cbd666 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:22.599300 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff0c0c-677a-419e-bec0-7857d9cbd666 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:22.599730 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff0c0c-677a-419e-bec0-7857d9cbd666 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:22.600015 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff0c0c-677a-419e-bec0-7857d9cbd666 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:22.622326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff0c0c-677a-419e-bec0-7857d9cbd666 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:22.624361 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff0c0c-677a-419e-bec0-7857d9cbd666 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:22.647944 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1072/2163] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:36:22 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:36:22.689711 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eaaf72-2eb4-4b98-a6d6-4f7206cfa33c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:22.691508 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eaaf72-2eb4-4b98-a6d6-4f7206cfa33c service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:22.710916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1092/2164] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:36:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:36:22.938174 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:23.313293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:23.691283 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff0c0c-677a-419e-bec0-7857d9cbd666 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:23.825750 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff0c0c-677a-419e-bec0-7857d9cbd666 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:23.840734 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04edb5-8e70-4a53-a5e1-d2597c792707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:23.841573 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04edb5-8e70-4a53-a5e1-d2597c792707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:23.842017 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04edb5-8e70-4a53-a5e1-d2597c792707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:23.842456 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04edb5-8e70-4a53-a5e1-d2597c792707 admin admin] PATH_INFO: `/v3/users/78d83b62d35a4b73a8bedcba9b13b93f` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:23.846755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04edb5-8e70-4a53-a5e1-d2597c792707 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78d83b62d35a4b73a8bedcba9b13b93f)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:23.849495 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04edb5-8e70-4a53-a5e1-d2597c792707 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:24.201714 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7e04edb5-8e70-4a53-a5e1-d2597c792707 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78d83b62d35a4b73a8bedcba9b13b93f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:24.203042 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7e04edb5-8e70-4a53-a5e1-d2597c792707 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78d83b62d35a4b73a8bedcba9b13b93f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:24.487227 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81229501-a7bd-45b7-af5d-12ea1b88821e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:24.492007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81229501-a7bd-45b7-af5d-12ea1b88821e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:24.492007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81229501-a7bd-45b7-af5d-12ea1b88821e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:24.549444 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-81229501-a7bd-45b7-af5d-12ea1b88821e None None] MFA Rules not processed for user `b1a5d200007a4474a2a25d2ae0915c14`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:24.551745 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-81229501-a7bd-45b7-af5d-12ea1b88821e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:24.722421 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1093/2165] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:24.728087 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81229501-a7bd-45b7-af5d-12ea1b88821e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:24.729877 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-81229501-a7bd-45b7-af5d-12ea1b88821e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:24.763998 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81229501-a7bd-45b7-af5d-12ea1b88821e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:24.774446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e250280-704f-4168-8f13-8fb1d37abcb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:24.776047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e250280-704f-4168-8f13-8fb1d37abcb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:24.776476 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e250280-704f-4168-8f13-8fb1d37abcb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:24.776857 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e250280-704f-4168-8f13-8fb1d37abcb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:24.779558 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e250280-704f-4168-8f13-8fb1d37abcb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:24.779906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e250280-704f-4168-8f13-8fb1d37abcb6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:24.894417 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7e04edb5-8e70-4a53-a5e1-d2597c792707 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78d83b62d35a4b73a8bedcba9b13b93f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:24.905540 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7e04edb5-8e70-4a53-a5e1-d2597c792707 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78d83b62d35a4b73a8bedcba9b13b93f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:24.907027 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7e04edb5-8e70-4a53-a5e1-d2597c792707 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78d83b62d35a4b73a8bedcba9b13b93f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:24.947017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1094/2166] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:36:24 2022] POST /identity/v3/projects => generated 371 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:36:24.951558 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e250280-704f-4168-8f13-8fb1d37abcb6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:24.955211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6e250280-704f-4168-8f13-8fb1d37abcb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:24.987336 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1073/2167] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:36:23 2022] DELETE /identity/v3/users/78d83b62d35a4b73a8bedcba9b13b93f => generated 0 bytes in 1298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:36:24.989364 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04edb5-8e70-4a53-a5e1-d2597c792707 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:24.991440 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7e04edb5-8e70-4a53-a5e1-d2597c792707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:25.053684 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e250280-704f-4168-8f13-8fb1d37abcb6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:25.130736 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04edb5-8e70-4a53-a5e1-d2597c792707 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:25.138985 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe07b4-6856-477b-826f-8b9902069205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:25.146099 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe07b4-6856-477b-826f-8b9902069205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:25.146099 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe07b4-6856-477b-826f-8b9902069205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:25.146099 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe07b4-6856-477b-826f-8b9902069205 admin admin] PATH_INFO: `/v3/users/d3a0c7877b0841b2b253b1ca01ba69c9` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:25.148122 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe07b4-6856-477b-826f-8b9902069205 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3a0c7877b0841b2b253b1ca01ba69c9)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:25.150364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58e765-2664-4a97-84e8-e01f8184c0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:25.151077 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58e765-2664-4a97-84e8-e01f8184c0bc admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:25.151364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58e765-2664-4a97-84e8-e01f8184c0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:25.151665 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58e765-2664-4a97-84e8-e01f8184c0bc admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:25.152369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd58e765-2664-4a97-84e8-e01f8184c0bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:25.152683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe07b4-6856-477b-826f-8b9902069205 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:25.154302 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd58e765-2664-4a97-84e8-e01f8184c0bc admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:25.361736 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1095/2168] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:36:24 2022] POST /identity/v3/users => generated 292 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:36:25.361736 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64d593-0e65-4509-8a68-7ca06e6e3999 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:25.361736 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64d593-0e65-4509-8a68-7ca06e6e3999 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:25.361736 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64d593-0e65-4509-8a68-7ca06e6e3999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:25.424179 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ce64d593-0e65-4509-8a68-7ca06e6e3999 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:25.428481 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ce64d593-0e65-4509-8a68-7ca06e6e3999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:25.436801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1096/2169] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:25.438113 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64d593-0e65-4509-8a68-7ca06e6e3999 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:25.453393 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64d593-0e65-4509-8a68-7ca06e6e3999 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:25.478284 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f8cba-1299-4bcf-ad6f-f9bb67a33b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:25.480213 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f8cba-1299-4bcf-ad6f-f9bb67a33b4f admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:25.480898 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f8cba-1299-4bcf-ad6f-f9bb67a33b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:25.481767 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f8cba-1299-4bcf-ad6f-f9bb67a33b4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:25.486053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f8cba-1299-4bcf-ad6f-f9bb67a33b4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:25.490572 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f8cba-1299-4bcf-ad6f-f9bb67a33b4f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:25.514709 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cbbe07b4-6856-477b-826f-8b9902069205 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3a0c7877b0841b2b253b1ca01ba69c9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:25.515369 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cbbe07b4-6856-477b-826f-8b9902069205 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3a0c7877b0841b2b253b1ca01ba69c9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:25.525941 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1097/2170] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:36:25.525941 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f8cba-1299-4bcf-ad6f-f9bb67a33b4f admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:25.545601 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b61f8cba-1299-4bcf-ad6f-f9bb67a33b4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:25.656302 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cbbe07b4-6856-477b-826f-8b9902069205 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3a0c7877b0841b2b253b1ca01ba69c9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:25.667701 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cbbe07b4-6856-477b-826f-8b9902069205 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3a0c7877b0841b2b253b1ca01ba69c9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:25.671071 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-cbbe07b4-6856-477b-826f-8b9902069205 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3a0c7877b0841b2b253b1ca01ba69c9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:25.723940 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1074/2171] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:36:24 2022] DELETE /identity/v3/users/d3a0c7877b0841b2b253b1ca01ba69c9 => generated 0 bytes in 735 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:25.728574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe07b4-6856-477b-826f-8b9902069205 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:25.730920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cbbe07b4-6856-477b-826f-8b9902069205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:25.807771 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f8cba-1299-4bcf-ad6f-f9bb67a33b4f admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:25.914284 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc498c-2ba7-4482-aedb-6c4969f575dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:25.915540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc498c-2ba7-4482-aedb-6c4969f575dc service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:25.916012 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc498c-2ba7-4482-aedb-6c4969f575dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:25.916453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc498c-2ba7-4482-aedb-6c4969f575dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:25.921561 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe07b4-6856-477b-826f-8b9902069205 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:25.922016 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a7cc498c-2ba7-4482-aedb-6c4969f575dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:25.929332 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a8c70-10bc-4629-aa49-1aeab94a5f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:25.930601 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a8c70-10bc-4629-aa49-1aeab94a5f9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:25.931051 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a8c70-10bc-4629-aa49-1aeab94a5f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:25.931551 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a8c70-10bc-4629-aa49-1aeab94a5f9c admin admin] PATH_INFO: `/v3/projects/3c9e3028b2404fc09a0a270fab412a09/users/23f851892cfa41d08c39d4929267946f/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:25.951311 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a8c70-10bc-4629-aa49-1aeab94a5f9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c9e3028b2404fc09a0a270fab412a09, user_id=23f851892cfa41d08c39d4929267946f, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:25.955849 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a8c70-10bc-4629-aa49-1aeab94a5f9c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:25.957407 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc498c-2ba7-4482-aedb-6c4969f575dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:25.960836 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc498c-2ba7-4482-aedb-6c4969f575dc service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:25.977313 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1098/2172] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:36:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:36:25.980181 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7aa6e-cc83-4d70-9f2e-8b45ac727371 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:25.980689 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7aa6e-cc83-4d70-9f2e-8b45ac727371 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:25.981807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7aa6e-cc83-4d70-9f2e-8b45ac727371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:26.008828 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b0d7aa6e-cc83-4d70-9f2e-8b45ac727371 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:26.010557 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b0d7aa6e-cc83-4d70-9f2e-8b45ac727371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:26.020945 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1099/2173] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:26.052486 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7aa6e-cc83-4d70-9f2e-8b45ac727371 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:26.054936 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b0d7aa6e-cc83-4d70-9f2e-8b45ac727371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:26.077933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1075/2174] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:36:25 2022] PUT /identity/v3/projects/3c9e3028b2404fc09a0a270fab412a09/users/23f851892cfa41d08c39d4929267946f/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:26.079289 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a8c70-10bc-4629-aa49-1aeab94a5f9c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:26.094689 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7aa6e-cc83-4d70-9f2e-8b45ac727371 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:26.206168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895ad04-b533-4567-83df-ddadd7ba1eb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:26.208702 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895ad04-b533-4567-83df-ddadd7ba1eb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:26.209403 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895ad04-b533-4567-83df-ddadd7ba1eb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:26.210057 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f895ad04-b533-4567-83df-ddadd7ba1eb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:26.242120 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a8c70-10bc-4629-aa49-1aeab94a5f9c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:26.247507 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc61db9-b1ed-4411-a809-b1d2f9d03071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:26.248558 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc61db9-b1ed-4411-a809-b1d2f9d03071 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:26.249054 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc61db9-b1ed-4411-a809-b1d2f9d03071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:26.249505 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc61db9-b1ed-4411-a809-b1d2f9d03071 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:26.250423 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc61db9-b1ed-4411-a809-b1d2f9d03071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:26.253252 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc61db9-b1ed-4411-a809-b1d2f9d03071 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:26.267494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1076/2175] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:36:26.273774 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc61db9-b1ed-4411-a809-b1d2f9d03071 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:26.290048 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc61db9-b1ed-4411-a809-b1d2f9d03071 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:26.300666 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aac777-3759-44a9-b378-730c223edeb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:26.300666 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aac777-3759-44a9-b378-730c223edeb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:26.300666 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aac777-3759-44a9-b378-730c223edeb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:26.300666 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aac777-3759-44a9-b378-730c223edeb1 admin admin] PATH_INFO: `/v3/projects/3c9e3028b2404fc09a0a270fab412a09/users/23f851892cfa41d08c39d4929267946f/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:26.303409 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aac777-3759-44a9-b378-730c223edeb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c9e3028b2404fc09a0a270fab412a09, user_id=23f851892cfa41d08c39d4929267946f, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:26.307753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895ad04-b533-4567-83df-ddadd7ba1eb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:26.309306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f895ad04-b533-4567-83df-ddadd7ba1eb9 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:26.313528 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aac777-3759-44a9-b378-730c223edeb1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:26.324003 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1100/2176] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:36:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:36:26.345140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1077/2177] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:36:26 2022] PUT /identity/v3/projects/3c9e3028b2404fc09a0a270fab412a09/users/23f851892cfa41d08c39d4929267946f/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:36:27.254033 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895ad04-b533-4567-83df-ddadd7ba1eb9 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:27.256388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f895ad04-b533-4567-83df-ddadd7ba1eb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:27.316702 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f895ad04-b533-4567-83df-ddadd7ba1eb9 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:27.388370 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:27.482514 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8746917-e9bb-4b9e-bb1b-a4a0d2aa1157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:27.483656 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8746917-e9bb-4b9e-bb1b-a4a0d2aa1157 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:27.484047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8746917-e9bb-4b9e-bb1b-a4a0d2aa1157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:27.484425 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8746917-e9bb-4b9e-bb1b-a4a0d2aa1157 admin admin] PATH_INFO: `/v3/projects/6d289d6f2a0d44e4b070c5ea6bbec0d3` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:27.486007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8746917-e9bb-4b9e-bb1b-a4a0d2aa1157 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d289d6f2a0d44e4b070c5ea6bbec0d3)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:27.489029 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8746917-e9bb-4b9e-bb1b-a4a0d2aa1157 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:27.643572 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:27.703921 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d8746917-e9bb-4b9e-bb1b-a4a0d2aa1157 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d289d6f2a0d44e4b070c5ea6bbec0d3', 'request_id': 'req-d8746917-e9bb-4b9e-bb1b-a4a0d2aa1157'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:27.740497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aac777-3759-44a9-b378-730c223edeb1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:27.743913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c3aac777-3759-44a9-b378-730c223edeb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:27.764491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aac777-3759-44a9-b378-730c223edeb1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:27.788417 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d8746917-e9bb-4b9e-bb1b-a4a0d2aa1157 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d289d6f2a0d44e4b070c5ea6bbec0d3', 'request_id': 'req-d8746917-e9bb-4b9e-bb1b-a4a0d2aa1157'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:27.799796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d8746917-e9bb-4b9e-bb1b-a4a0d2aa1157 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d289d6f2a0d44e4b070c5ea6bbec0d3', 'request_id': 'req-d8746917-e9bb-4b9e-bb1b-a4a0d2aa1157'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:27.825353 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d8746917-e9bb-4b9e-bb1b-a4a0d2aa1157 admin admin] The token cache is being invalidate because project 6d289d6f2a0d44e4b070c5ea6bbec0d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:36:27.825985 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d8746917-e9bb-4b9e-bb1b-a4a0d2aa1157 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:27.828257 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d8746917-e9bb-4b9e-bb1b-a4a0d2aa1157 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:27.833317 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1101/2178] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:36:27 2022] DELETE /identity/v3/projects/6d289d6f2a0d44e4b070c5ea6bbec0d3 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:27.839505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8746917-e9bb-4b9e-bb1b-a4a0d2aa1157 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:27.843044 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d8746917-e9bb-4b9e-bb1b-a4a0d2aa1157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:27.956741 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acc558-9394-4223-ae6e-a84d630cf04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:27.958111 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acc558-9394-4223-ae6e-a84d630cf04d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:27.958405 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acc558-9394-4223-ae6e-a84d630cf04d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:27.958880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acc558-9394-4223-ae6e-a84d630cf04d admin admin] PATH_INFO: `/v3/projects/0908533e5a3345ecaeac47b5bd3526e8` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:27.960431 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65acc558-9394-4223-ae6e-a84d630cf04d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0908533e5a3345ecaeac47b5bd3526e8)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:27.962425 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65acc558-9394-4223-ae6e-a84d630cf04d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:27.975123 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8746917-e9bb-4b9e-bb1b-a4a0d2aa1157 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:27.980449 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b029a-60dc-40a7-a3f0-4d60b4dda7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:27.981577 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b029a-60dc-40a7-a3f0-4d60b4dda7d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:27.982033 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b029a-60dc-40a7-a3f0-4d60b4dda7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:27.982432 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b029a-60dc-40a7-a3f0-4d60b4dda7d8 admin admin] PATH_INFO: `/v3/users/96d2252a94a6431995f5d85abc65a491` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:27.984229 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675b029a-60dc-40a7-a3f0-4d60b4dda7d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96d2252a94a6431995f5d85abc65a491)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:27.986989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675b029a-60dc-40a7-a3f0-4d60b4dda7d8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:28.224135 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-65acc558-9394-4223-ae6e-a84d630cf04d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0908533e5a3345ecaeac47b5bd3526e8', 'request_id': 'req-65acc558-9394-4223-ae6e-a84d630cf04d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:28.236874 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-65acc558-9394-4223-ae6e-a84d630cf04d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0908533e5a3345ecaeac47b5bd3526e8', 'request_id': 'req-65acc558-9394-4223-ae6e-a84d630cf04d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:28.248288 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-65acc558-9394-4223-ae6e-a84d630cf04d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0908533e5a3345ecaeac47b5bd3526e8', 'request_id': 'req-65acc558-9394-4223-ae6e-a84d630cf04d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:28.376034 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-675b029a-60dc-40a7-a3f0-4d60b4dda7d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96d2252a94a6431995f5d85abc65a491'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:28.379162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-675b029a-60dc-40a7-a3f0-4d60b4dda7d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96d2252a94a6431995f5d85abc65a491'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:28.401615 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-675b029a-60dc-40a7-a3f0-4d60b4dda7d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96d2252a94a6431995f5d85abc65a491'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:28.406202 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-65acc558-9394-4223-ae6e-a84d630cf04d admin admin] The token cache is being invalidate because project 0908533e5a3345ecaeac47b5bd3526e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:36:28.406202 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-65acc558-9394-4223-ae6e-a84d630cf04d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:28.419829 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-65acc558-9394-4223-ae6e-a84d630cf04d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:28.430756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1078/2179] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:36:27 2022] DELETE /identity/v3/projects/0908533e5a3345ecaeac47b5bd3526e8 => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:36:28.434051 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acc558-9394-4223-ae6e-a84d630cf04d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:28.436687 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-65acc558-9394-4223-ae6e-a84d630cf04d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:28.513585 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-675b029a-60dc-40a7-a3f0-4d60b4dda7d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96d2252a94a6431995f5d85abc65a491'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:28.514294 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-675b029a-60dc-40a7-a3f0-4d60b4dda7d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96d2252a94a6431995f5d85abc65a491'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:28.569735 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1102/2180] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:36:27 2022] DELETE /identity/v3/users/96d2252a94a6431995f5d85abc65a491 => generated 0 bytes in 735 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:28.634532 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acc558-9394-4223-ae6e-a84d630cf04d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:28.718742 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30766a2d-5221-4778-b92c-0c3cfff209eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:28.720127 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30766a2d-5221-4778-b92c-0c3cfff209eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:28.720127 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30766a2d-5221-4778-b92c-0c3cfff209eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:28.720463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30766a2d-5221-4778-b92c-0c3cfff209eb admin admin] PATH_INFO: `/v3/users/99133f76012742b2ae1765b2f4662f94` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:28.725265 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30766a2d-5221-4778-b92c-0c3cfff209eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=99133f76012742b2ae1765b2f4662f94)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:28.727415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30766a2d-5221-4778-b92c-0c3cfff209eb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:29.039868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-30766a2d-5221-4778-b92c-0c3cfff209eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99133f76012742b2ae1765b2f4662f94'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:29.040958 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-30766a2d-5221-4778-b92c-0c3cfff209eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99133f76012742b2ae1765b2f4662f94'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:29.151888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-30766a2d-5221-4778-b92c-0c3cfff209eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99133f76012742b2ae1765b2f4662f94'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:29.165962 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-30766a2d-5221-4778-b92c-0c3cfff209eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99133f76012742b2ae1765b2f4662f94'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:29.167660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-30766a2d-5221-4778-b92c-0c3cfff209eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99133f76012742b2ae1765b2f4662f94'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:29.253171 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1079/2181] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:36:28 2022] DELETE /identity/v3/users/99133f76012742b2ae1765b2f4662f94 => generated 0 bytes in 820 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:29.405697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b029a-60dc-40a7-a3f0-4d60b4dda7d8 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:29.408300 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-675b029a-60dc-40a7-a3f0-4d60b4dda7d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:29.432804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b029a-60dc-40a7-a3f0-4d60b4dda7d8 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:29.539208 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0695e52-e474-4496-850a-27bf5c8f7eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:29.539975 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0695e52-e474-4496-850a-27bf5c8f7eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:29.540324 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0695e52-e474-4496-850a-27bf5c8f7eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:29.542175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0695e52-e474-4496-850a-27bf5c8f7eb2 admin admin] PATH_INFO: `/v3/projects/986b61ce70d74220a5e945a34e3e1de0` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:29.550165 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0695e52-e474-4496-850a-27bf5c8f7eb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=986b61ce70d74220a5e945a34e3e1de0)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:29.550165 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0695e52-e474-4496-850a-27bf5c8f7eb2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:29.729378 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b0695e52-e474-4496-850a-27bf5c8f7eb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '986b61ce70d74220a5e945a34e3e1de0', 'request_id': 'req-b0695e52-e474-4496-850a-27bf5c8f7eb2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:29.885029 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b0695e52-e474-4496-850a-27bf5c8f7eb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '986b61ce70d74220a5e945a34e3e1de0', 'request_id': 'req-b0695e52-e474-4496-850a-27bf5c8f7eb2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:29.898300 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b0695e52-e474-4496-850a-27bf5c8f7eb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '986b61ce70d74220a5e945a34e3e1de0', 'request_id': 'req-b0695e52-e474-4496-850a-27bf5c8f7eb2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:29.922615 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b0695e52-e474-4496-850a-27bf5c8f7eb2 admin admin] The token cache is being invalidate because project 986b61ce70d74220a5e945a34e3e1de0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:36:29.923430 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b0695e52-e474-4496-850a-27bf5c8f7eb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:29.924923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b0695e52-e474-4496-850a-27bf5c8f7eb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:29.928828 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1103/2182] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:36:29 2022] DELETE /identity/v3/projects/986b61ce70d74220a5e945a34e3e1de0 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:29.936102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f024e0a-cc7f-47d0-a4a1-19dd7c8ebc1b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:29.937026 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f024e0a-cc7f-47d0-a4a1-19dd7c8ebc1b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:29.937809 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f024e0a-cc7f-47d0-a4a1-19dd7c8ebc1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:29.976576 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7f024e0a-cc7f-47d0-a4a1-19dd7c8ebc1b None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:29.983544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7f024e0a-cc7f-47d0-a4a1-19dd7c8ebc1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:30.045223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0695e52-e474-4496-850a-27bf5c8f7eb2 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:30.048304 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b0695e52-e474-4496-850a-27bf5c8f7eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:30.187090 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0695e52-e474-4496-850a-27bf5c8f7eb2 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:30.195704 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1080/2183] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:36:30.208500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df377c-43c0-4967-82bc-e8e4c9c4a8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:30.210717 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d03d95-214e-46ff-9a1b-30fba081d79e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:30.211942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df377c-43c0-4967-82bc-e8e4c9c4a8dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:30.212273 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d03d95-214e-46ff-9a1b-30fba081d79e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:30.212534 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df377c-43c0-4967-82bc-e8e4c9c4a8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:30.212714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df377c-43c0-4967-82bc-e8e4c9c4a8dc admin admin] PATH_INFO: `/v3/projects/e3ed410ca3a848baad97986417086b07` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:30.214045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d03d95-214e-46ff-9a1b-30fba081d79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:30.214878 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df377c-43c0-4967-82bc-e8e4c9c4a8dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3ed410ca3a848baad97986417086b07)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:30.217825 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df377c-43c0-4967-82bc-e8e4c9c4a8dc admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:30.279584 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f3d03d95-214e-46ff-9a1b-30fba081d79e None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:30.280850 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f3d03d95-214e-46ff-9a1b-30fba081d79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:30.302183 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1081/2184] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:30.308875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d03d95-214e-46ff-9a1b-30fba081d79e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:30.328839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d03d95-214e-46ff-9a1b-30fba081d79e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:30.352687 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb06041-37ad-4999-8892-e896f785254f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:30.353748 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb06041-37ad-4999-8892-e896f785254f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:30.354198 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb06041-37ad-4999-8892-e896f785254f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:30.354483 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb06041-37ad-4999-8892-e896f785254f admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:30.355362 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb06041-37ad-4999-8892-e896f785254f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:36:30.355626 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb06041-37ad-4999-8892-e896f785254f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:30.365377 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb06041-37ad-4999-8892-e896f785254f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:30.392179 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:30.416906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b1df377c-43c0-4967-82bc-e8e4c9c4a8dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3ed410ca3a848baad97986417086b07', 'request_id': 'req-b1df377c-43c0-4967-82bc-e8e4c9c4a8dc'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:30.419422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1082/2185] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:36:30.426658 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb06041-37ad-4999-8892-e896f785254f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:30.443746 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb06041-37ad-4999-8892-e896f785254f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:30.540370 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b1df377c-43c0-4967-82bc-e8e4c9c4a8dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3ed410ca3a848baad97986417086b07', 'request_id': 'req-b1df377c-43c0-4967-82bc-e8e4c9c4a8dc'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:30.548763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b1df377c-43c0-4967-82bc-e8e4c9c4a8dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3ed410ca3a848baad97986417086b07', 'request_id': 'req-b1df377c-43c0-4967-82bc-e8e4c9c4a8dc'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:30.565173 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b1df377c-43c0-4967-82bc-e8e4c9c4a8dc admin admin] The token cache is being invalidate because project e3ed410ca3a848baad97986417086b07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:36:30.565998 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b1df377c-43c0-4967-82bc-e8e4c9c4a8dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:30.567076 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b1df377c-43c0-4967-82bc-e8e4c9c4a8dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:30.569561 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1104/2186] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:36:30 2022] DELETE /identity/v3/projects/e3ed410ca3a848baad97986417086b07 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:30.575371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abbc7b-8840-4d30-81d8-88b9e6e7c3fc None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:30.576713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abbc7b-8840-4d30-81d8-88b9e6e7c3fc None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:30.577180 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abbc7b-8840-4d30-81d8-88b9e6e7c3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:30.582931 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0024c-dacd-44f9-9b09-40868628de85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:30.584227 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0024c-dacd-44f9-9b09-40868628de85 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:30.584774 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0024c-dacd-44f9-9b09-40868628de85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:30.585242 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b0024c-dacd-44f9-9b09-40868628de85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:30.586730 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0024c-dacd-44f9-9b09-40868628de85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:30.593062 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b0024c-dacd-44f9-9b09-40868628de85 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:30.604596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-86abbc7b-8840-4d30-81d8-88b9e6e7c3fc None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:30.606263 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-86abbc7b-8840-4d30-81d8-88b9e6e7c3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:30.629463 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1105/2187] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:30.632101 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffebdd2-86ad-4d8f-b884-901b7c179d81 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:30.632594 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffebdd2-86ad-4d8f-b884-901b7c179d81 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:30.633215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffebdd2-86ad-4d8f-b884-901b7c179d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:30.664738 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1ffebdd2-86ad-4d8f-b884-901b7c179d81 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:30.664738 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1ffebdd2-86ad-4d8f-b884-901b7c179d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:30.672242 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1106/2188] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:30.677065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffebdd2-86ad-4d8f-b884-901b7c179d81 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:30.690648 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffebdd2-86ad-4d8f-b884-901b7c179d81 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:30.699314 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042a763-3a5d-4e3c-841b-5dc1dab843d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:30.700772 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042a763-3a5d-4e3c-841b-5dc1dab843d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:30.701289 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042a763-3a5d-4e3c-841b-5dc1dab843d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:30.702105 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042a763-3a5d-4e3c-841b-5dc1dab843d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:30.707704 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042a763-3a5d-4e3c-841b-5dc1dab843d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:36:30.707704 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042a763-3a5d-4e3c-841b-5dc1dab843d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:30.708861 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042a763-3a5d-4e3c-841b-5dc1dab843d3 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:30.729921 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1083/2189] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:36:30 2022] POST /identity/v3/projects => generated 371 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:36:30.735627 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1107/2190] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:36:30.736759 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0024c-dacd-44f9-9b09-40868628de85 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:30.739094 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-12b0024c-dacd-44f9-9b09-40868628de85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:30.741636 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042a763-3a5d-4e3c-841b-5dc1dab843d3 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:30.763810 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042a763-3a5d-4e3c-841b-5dc1dab843d3 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:30.922064 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b0024c-dacd-44f9-9b09-40868628de85 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:30.937005 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4f8ae-30a1-48e2-8156-b91e41f6244d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:30.938542 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229cb45-925b-4859-9df4-eca963042ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:30.939562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4f8ae-30a1-48e2-8156-b91e41f6244d admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:30.939683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229cb45-925b-4859-9df4-eca963042ae3 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:30.940187 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229cb45-925b-4859-9df4-eca963042ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:30.940734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4f8ae-30a1-48e2-8156-b91e41f6244d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:30.941962 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4f8ae-30a1-48e2-8156-b91e41f6244d admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:30.942264 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229cb45-925b-4859-9df4-eca963042ae3 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:30.943757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4f8ae-30a1-48e2-8156-b91e41f6244d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:30.945336 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f229cb45-925b-4859-9df4-eca963042ae3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:30.947773 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4f8ae-30a1-48e2-8156-b91e41f6244d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:30.949348 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f229cb45-925b-4859-9df4-eca963042ae3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:31.088000 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1084/2191] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:36:30 2022] POST /identity/v3/users => generated 286 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:36:31.093704 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229cb45-925b-4859-9df4-eca963042ae3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:31.112774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1108/2192] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:36:30 2022] POST /identity/v3/projects => generated 369 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:36:31.114505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4f8ae-30a1-48e2-8156-b91e41f6244d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:31.117174 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f229cb45-925b-4859-9df4-eca963042ae3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:31.196890 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d1951-b220-4b67-80eb-614dd0a4b839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:31.198139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d1951-b220-4b67-80eb-614dd0a4b839 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:31.198474 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d1951-b220-4b67-80eb-614dd0a4b839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:31.198922 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d1951-b220-4b67-80eb-614dd0a4b839 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:31.199517 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d1951-b220-4b67-80eb-614dd0a4b839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:31.201415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d1951-b220-4b67-80eb-614dd0a4b839 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:31.218002 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4f8ae-30a1-48e2-8156-b91e41f6244d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:31.218455 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1085/2193] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:36:31.222144 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d1951-b220-4b67-80eb-614dd0a4b839 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:31.225694 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757f393-0a7a-4cde-b1be-10995b8a9a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:31.226587 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757f393-0a7a-4cde-b1be-10995b8a9a27 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:31.226743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757f393-0a7a-4cde-b1be-10995b8a9a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:31.227024 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757f393-0a7a-4cde-b1be-10995b8a9a27 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:31.227803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757f393-0a7a-4cde-b1be-10995b8a9a27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:31.230806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e757f393-0a7a-4cde-b1be-10995b8a9a27 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:31.238791 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d1951-b220-4b67-80eb-614dd0a4b839 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:31.249000 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6134d2-ec40-4ea5-853e-3a7a22f5e202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:31.260087 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6134d2-ec40-4ea5-853e-3a7a22f5e202 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:31.260087 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6134d2-ec40-4ea5-853e-3a7a22f5e202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:31.260087 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6134d2-ec40-4ea5-853e-3a7a22f5e202 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:31.260087 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6134d2-ec40-4ea5-853e-3a7a22f5e202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:31.260087 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6134d2-ec40-4ea5-853e-3a7a22f5e202 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:31.277314 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1086/2194] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:36:31.282311 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6134d2-ec40-4ea5-853e-3a7a22f5e202 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:31.304467 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6134d2-ec40-4ea5-853e-3a7a22f5e202 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:31.311907 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2979b535-8e33-4a02-817e-762a8013ccda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:31.313268 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979b535-8e33-4a02-817e-762a8013ccda admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:31.313268 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979b535-8e33-4a02-817e-762a8013ccda admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:31.313519 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2979b535-8e33-4a02-817e-762a8013ccda admin admin] PATH_INFO: `/v3/projects/8dd8bfff09ae42ac94de0c9ae7d13fdd/users/a21934ec65eb427c9a5c3935a019dd73/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:31.366519 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1109/2195] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:36:31 2022] POST /identity/v3/users => generated 285 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:36:31.373768 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757f393-0a7a-4cde-b1be-10995b8a9a27 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:31.408487 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2979b535-8e33-4a02-817e-762a8013ccda admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dd8bfff09ae42ac94de0c9ae7d13fdd, user_id=a21934ec65eb427c9a5c3935a019dd73, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:31.416839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2979b535-8e33-4a02-817e-762a8013ccda admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:31.439652 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757f393-0a7a-4cde-b1be-10995b8a9a27 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:31.458659 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe0ee14-7108-4d38-81f1-b4272e203a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:31.468610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0ee14-7108-4d38-81f1-b4272e203a48 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:31.468610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0ee14-7108-4d38-81f1-b4272e203a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:31.468610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0ee14-7108-4d38-81f1-b4272e203a48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:31.468610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe0ee14-7108-4d38-81f1-b4272e203a48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:31.477030 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe0ee14-7108-4d38-81f1-b4272e203a48 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:31.503189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1110/2196] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:36:31.511914 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe0ee14-7108-4d38-81f1-b4272e203a48 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:31.525317 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe0ee14-7108-4d38-81f1-b4272e203a48 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:31.541575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1087/2197] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:36:31 2022] PUT /identity/v3/projects/8dd8bfff09ae42ac94de0c9ae7d13fdd/users/a21934ec65eb427c9a5c3935a019dd73/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:31.554320 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92ce56-0141-49b7-ae0a-703ae3c31f5f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:31.554320 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92ce56-0141-49b7-ae0a-703ae3c31f5f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:31.554320 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92ce56-0141-49b7-ae0a-703ae3c31f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:31.585871 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:31.610741 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4c92ce56-0141-49b7-ae0a-703ae3c31f5f None None] MFA Rules not processed for user `a21934ec65eb427c9a5c3935a019dd73`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:31.634013 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4c92ce56-0141-49b7-ae0a-703ae3c31f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:31.748180 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d3097-7e71-411b-b80b-220b68572192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:31.754425 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d3097-7e71-411b-b80b-220b68572192 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:31.755945 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d3097-7e71-411b-b80b-220b68572192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:31.755945 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d3097-7e71-411b-b80b-220b68572192 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:31.756260 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d3097-7e71-411b-b80b-220b68572192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:31.758653 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d3097-7e71-411b-b80b-220b68572192 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:31.808301 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1111/2198] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:36:31.811808 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d3097-7e71-411b-b80b-220b68572192 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:31.849357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d3097-7e71-411b-b80b-220b68572192 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:31.860158 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d393bab-eb2a-414e-a04d-13b8003aee7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:31.861151 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d393bab-eb2a-414e-a04d-13b8003aee7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:31.861422 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d393bab-eb2a-414e-a04d-13b8003aee7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:31.861694 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d393bab-eb2a-414e-a04d-13b8003aee7a admin admin] PATH_INFO: `/v3/projects/1933c5b6c74d4a53a6782f833c617b2f/users/6b485c044e984d90a76ee17abe5c3eda/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:31.885269 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1088/2199] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:31.898760 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92ce56-0141-49b7-ae0a-703ae3c31f5f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:31.901550 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4c92ce56-0141-49b7-ae0a-703ae3c31f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:31.910925 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d393bab-eb2a-414e-a04d-13b8003aee7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1933c5b6c74d4a53a6782f833c617b2f, user_id=6b485c044e984d90a76ee17abe5c3eda, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:31.922164 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d393bab-eb2a-414e-a04d-13b8003aee7a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:32.032282 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92ce56-0141-49b7-ae0a-703ae3c31f5f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:32.038284 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed0725-dfb2-404f-bf56-7bff72ca9c44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:32.039634 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed0725-dfb2-404f-bf56-7bff72ca9c44 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:32.040205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed0725-dfb2-404f-bf56-7bff72ca9c44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:32.040557 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed0725-dfb2-404f-bf56-7bff72ca9c44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:32.067015 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ed0725-dfb2-404f-bf56-7bff72ca9c44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:32.067015 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ed0725-dfb2-404f-bf56-7bff72ca9c44 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:32.080795 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1089/2200] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:36:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:36:32.122036 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed0725-dfb2-404f-bf56-7bff72ca9c44 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:32.149701 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed0725-dfb2-404f-bf56-7bff72ca9c44 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:32.156520 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12dcef3-a147-4336-af5a-bb9a47a97dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:32.157544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dcef3-a147-4336-af5a-bb9a47a97dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:32.158394 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dcef3-a147-4336-af5a-bb9a47a97dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:32.158752 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dcef3-a147-4336-af5a-bb9a47a97dcd admin admin] PATH_INFO: `/v3/users/a21934ec65eb427c9a5c3935a019dd73` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:32.166359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12dcef3-a147-4336-af5a-bb9a47a97dcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a21934ec65eb427c9a5c3935a019dd73)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:32.170431 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12dcef3-a147-4336-af5a-bb9a47a97dcd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:32.243429 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1112/2201] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:36:31 2022] PUT /identity/v3/projects/1933c5b6c74d4a53a6782f833c617b2f/users/6b485c044e984d90a76ee17abe5c3eda/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:32.243429 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc52c1-ff82-48a4-a779-cdf9da7bd7a2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:32.243429 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc52c1-ff82-48a4-a779-cdf9da7bd7a2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:32.243429 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc52c1-ff82-48a4-a779-cdf9da7bd7a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:32.295926 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-bacc52c1-ff82-48a4-a779-cdf9da7bd7a2 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:32.298372 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bacc52c1-ff82-48a4-a779-cdf9da7bd7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:32.641572 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1113/2202] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:36:32.672325 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc52c1-ff82-48a4-a779-cdf9da7bd7a2 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:32.675013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bacc52c1-ff82-48a4-a779-cdf9da7bd7a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:32.935780 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc52c1-ff82-48a4-a779-cdf9da7bd7a2 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:32.955246 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b3964-7740-481a-8798-236fbb9b7f5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:32.956540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b3964-7740-481a-8798-236fbb9b7f5b service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:32.957131 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b3964-7740-481a-8798-236fbb9b7f5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:32.958535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b3964-7740-481a-8798-236fbb9b7f5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:32.978816 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42b3964-7740-481a-8798-236fbb9b7f5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:32.981425 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42b3964-7740-481a-8798-236fbb9b7f5b service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:32.998885 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1114/2203] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:36:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:36:33.006767 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0588e96-d6c6-4cd1-a8bb-c19d60c2cd6e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:33.006767 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0588e96-d6c6-4cd1-a8bb-c19d60c2cd6e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:33.006767 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0588e96-d6c6-4cd1-a8bb-c19d60c2cd6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:33.142919 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c12dcef3-a147-4336-af5a-bb9a47a97dcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a21934ec65eb427c9a5c3935a019dd73'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:33.144647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c12dcef3-a147-4336-af5a-bb9a47a97dcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a21934ec65eb427c9a5c3935a019dd73'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:33.172986 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d0588e96-d6c6-4cd1-a8bb-c19d60c2cd6e None None] MFA Rules not processed for user `d49604bffa3d4e89a631eefa6d4e4f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:33.187083 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d0588e96-d6c6-4cd1-a8bb-c19d60c2cd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:33.323278 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1115/2204] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 30 16:36:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:33.337467 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c12dcef3-a147-4336-af5a-bb9a47a97dcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a21934ec65eb427c9a5c3935a019dd73'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:33.364246 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c12dcef3-a147-4336-af5a-bb9a47a97dcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a21934ec65eb427c9a5c3935a019dd73'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:33.369045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c12dcef3-a147-4336-af5a-bb9a47a97dcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a21934ec65eb427c9a5c3935a019dd73'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:33.444533 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:33.451527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1090/2205] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:36:32 2022] DELETE /identity/v3/users/a21934ec65eb427c9a5c3935a019dd73 => generated 0 bytes in 1325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:36:33.453181 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375002b-ed50-4ecc-b763-1c3fdd2f4731 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:33.453526 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375002b-ed50-4ecc-b763-1c3fdd2f4731 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:33.453805 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375002b-ed50-4ecc-b763-1c3fdd2f4731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:33.477940 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12dcef3-a147-4336-af5a-bb9a47a97dcd admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:33.483634 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c12dcef3-a147-4336-af5a-bb9a47a97dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:33.490875 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c375002b-ed50-4ecc-b763-1c3fdd2f4731 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:33.492259 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c375002b-ed50-4ecc-b763-1c3fdd2f4731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:33.587785 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1116/2206] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:33.648035 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375002b-ed50-4ecc-b763-1c3fdd2f4731 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:33.650410 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c375002b-ed50-4ecc-b763-1c3fdd2f4731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:33.884644 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12dcef3-a147-4336-af5a-bb9a47a97dcd admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:33.895506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb6dc8-6dd3-42de-8561-3d909eab0c9d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988b917eea4b48ffae32e23b8555a5c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:33.895506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb6dc8-6dd3-42de-8561-3d909eab0c9d service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:33.895506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb6dc8-6dd3-42de-8561-3d909eab0c9d service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:33.895506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb6dc8-6dd3-42de-8561-3d909eab0c9d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:33.897383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0beb6dc8-6dd3-42de-8561-3d909eab0c9d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:33.905277 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375002b-ed50-4ecc-b763-1c3fdd2f4731 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:33.913629 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5d7e8-4a93-4066-9266-6d18992d88b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:33.914987 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5d7e8-4a93-4066-9266-6d18992d88b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:33.915609 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5d7e8-4a93-4066-9266-6d18992d88b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:33.916436 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5d7e8-4a93-4066-9266-6d18992d88b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:33.969386 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5d7e8-4a93-4066-9266-6d18992d88b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:33.971962 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c5d7e8-4a93-4066-9266-6d18992d88b0 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:33.992145 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb6dc8-6dd3-42de-8561-3d909eab0c9d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:33.994118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb6dc8-6dd3-42de-8561-3d909eab0c9d service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:34.011682 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1117/2207] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:36:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:36:34.038438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1091/2208] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 30 16:36:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 16:36:34.291986 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:34.632353 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:35.639035 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:36.450004 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:36.538357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f33e70-cbc3-4c55-a7b9-b31a893c2c98 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:36.538793 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f33e70-cbc3-4c55-a7b9-b31a893c2c98 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:36.539202 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f33e70-cbc3-4c55-a7b9-b31a893c2c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:36.570324 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb6dc8-6dd3-42de-8561-3d909eab0c9d service swift] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:36.571885 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0beb6dc8-6dd3-42de-8561-3d909eab0c9d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:36.608378 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb6dc8-6dd3-42de-8561-3d909eab0c9d service swift] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:36.614921 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-86f33e70-cbc3-4c55-a7b9-b31a893c2c98 None None] MFA Rules not processed for user `6b485c044e984d90a76ee17abe5c3eda`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:36.618283 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4591b5-13d3-40cf-a5f0-adc5ddda777c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:36.619340 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4591b5-13d3-40cf-a5f0-adc5ddda777c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:36.619725 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4591b5-13d3-40cf-a5f0-adc5ddda777c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:36.620131 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4591b5-13d3-40cf-a5f0-adc5ddda777c admin admin] PATH_INFO: `/v3/projects/8dd8bfff09ae42ac94de0c9ae7d13fdd` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:36.620969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-86f33e70-cbc3-4c55-a7b9-b31a893c2c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:36.622971 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4591b5-13d3-40cf-a5f0-adc5ddda777c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dd8bfff09ae42ac94de0c9ae7d13fdd)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:36.625198 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4591b5-13d3-40cf-a5f0-adc5ddda777c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:36.854811 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8f4591b5-13d3-40cf-a5f0-adc5ddda777c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dd8bfff09ae42ac94de0c9ae7d13fdd', 'request_id': 'req-8f4591b5-13d3-40cf-a5f0-adc5ddda777c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:36.886777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8f4591b5-13d3-40cf-a5f0-adc5ddda777c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dd8bfff09ae42ac94de0c9ae7d13fdd', 'request_id': 'req-8f4591b5-13d3-40cf-a5f0-adc5ddda777c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:36.887894 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1118/2209] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:36 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:36.918556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8f4591b5-13d3-40cf-a5f0-adc5ddda777c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dd8bfff09ae42ac94de0c9ae7d13fdd', 'request_id': 'req-8f4591b5-13d3-40cf-a5f0-adc5ddda777c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:36.920522 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f33e70-cbc3-4c55-a7b9-b31a893c2c98 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:36.936235 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-86f33e70-cbc3-4c55-a7b9-b31a893c2c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:37.016651 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f33e70-cbc3-4c55-a7b9-b31a893c2c98 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:37.064631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8f4591b5-13d3-40cf-a5f0-adc5ddda777c admin admin] The token cache is being invalidate because project 8dd8bfff09ae42ac94de0c9ae7d13fdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:36:37.064945 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8f4591b5-13d3-40cf-a5f0-adc5ddda777c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:37.065777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8f4591b5-13d3-40cf-a5f0-adc5ddda777c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:37.076728 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1092/2210] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:36:36 2022] DELETE /identity/v3/projects/8dd8bfff09ae42ac94de0c9ae7d13fdd => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:37.084278 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bb870-d77b-4536-87a0-f9b53eb12c85 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:37.084719 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bb870-d77b-4536-87a0-f9b53eb12c85 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:37.085427 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2bb870-d77b-4536-87a0-f9b53eb12c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:37.173756 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afcf4a-907d-410f-bd12-4dc2541212d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:37.174590 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afcf4a-907d-410f-bd12-4dc2541212d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:37.174983 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afcf4a-907d-410f-bd12-4dc2541212d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:37.175384 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afcf4a-907d-410f-bd12-4dc2541212d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:37.178395 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afcf4a-907d-410f-bd12-4dc2541212d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:37.180598 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afcf4a-907d-410f-bd12-4dc2541212d6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:37.203560 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ad2bb870-d77b-4536-87a0-f9b53eb12c85 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:37.212027 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ad2bb870-d77b-4536-87a0-f9b53eb12c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:37.238096 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1093/2211] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:37.246134 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83347b47-f637-43f6-9d86-f386d902c369 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:37.252551 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83347b47-f637-43f6-9d86-f386d902c369 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:37.252551 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83347b47-f637-43f6-9d86-f386d902c369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:37.284788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1119/2212] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:36:36 2022] POST /identity/v3/projects => generated 369 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:36:37.289540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afcf4a-907d-410f-bd12-4dc2541212d6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:37.294715 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-83347b47-f637-43f6-9d86-f386d902c369 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:37.296508 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-83347b47-f637-43f6-9d86-f386d902c369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:37.298084 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-88afcf4a-907d-410f-bd12-4dc2541212d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:37.441624 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:37.513767 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1094/2213] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:36:37.515372 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83347b47-f637-43f6-9d86-f386d902c369 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:37.532683 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afcf4a-907d-410f-bd12-4dc2541212d6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:37.545725 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6c81b-db72-402b-8180-47921cafa270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:37.546659 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6c81b-db72-402b-8180-47921cafa270 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:37.551008 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6c81b-db72-402b-8180-47921cafa270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:37.552632 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f6c81b-db72-402b-8180-47921cafa270 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:37.553346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83347b47-f637-43f6-9d86-f386d902c369 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:37.554176 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f6c81b-db72-402b-8180-47921cafa270 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:37.565295 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f6c81b-db72-402b-8180-47921cafa270 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:37.573131 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d6b057-ce21-4b18-adb7-2bfdef8c96f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:37.574471 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6b057-ce21-4b18-adb7-2bfdef8c96f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:37.574928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6b057-ce21-4b18-adb7-2bfdef8c96f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:37.575366 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d6b057-ce21-4b18-adb7-2bfdef8c96f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:37.576392 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d6b057-ce21-4b18-adb7-2bfdef8c96f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:36:37.576869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d6b057-ce21-4b18-adb7-2bfdef8c96f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:37.580933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d6b057-ce21-4b18-adb7-2bfdef8c96f3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:37.626701 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1095/2214] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:36:37.628647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d6b057-ce21-4b18-adb7-2bfdef8c96f3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:37.648630 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d6b057-ce21-4b18-adb7-2bfdef8c96f3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:37.670514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2da31-9185-45c6-a769-2c266bf7c746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:37.670514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2da31-9185-45c6-a769-2c266bf7c746 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:37.670514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2da31-9185-45c6-a769-2c266bf7c746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:37.670514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2da31-9185-45c6-a769-2c266bf7c746 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:37.674267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb2da31-9185-45c6-a769-2c266bf7c746 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:37.682895 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb2da31-9185-45c6-a769-2c266bf7c746 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:37.715166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1120/2215] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:36:37 2022] POST /identity/v3/users => generated 291 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:36:37.717102 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6c81b-db72-402b-8180-47921cafa270 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:37.741331 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f6c81b-db72-402b-8180-47921cafa270 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:37.749269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b8e0d-1c22-4267-9615-cadca3f0405b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:37.750589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b8e0d-1c22-4267-9615-cadca3f0405b admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:37.750690 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b8e0d-1c22-4267-9615-cadca3f0405b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:37.751500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b8e0d-1c22-4267-9615-cadca3f0405b admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:37.752343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b8e0d-1c22-4267-9615-cadca3f0405b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:37.755421 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b8e0d-1c22-4267-9615-cadca3f0405b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:37.780849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1121/2216] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:36:37.786024 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b8e0d-1c22-4267-9615-cadca3f0405b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:37.805035 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b8e0d-1c22-4267-9615-cadca3f0405b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:37.811418 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6dbbc5-d052-42c7-b701-8fe48fcfd308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:37.813204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6dbbc5-d052-42c7-b701-8fe48fcfd308 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:37.813499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6dbbc5-d052-42c7-b701-8fe48fcfd308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:37.813995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6dbbc5-d052-42c7-b701-8fe48fcfd308 admin admin] PATH_INFO: `/v3/projects/13971c73036e42dc801a51a5b70971b6/users/a9ded5a7330346b8b5492268654d828f/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:37.838462 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6dbbc5-d052-42c7-b701-8fe48fcfd308 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13971c73036e42dc801a51a5b70971b6, user_id=a9ded5a7330346b8b5492268654d828f, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:37.842076 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6dbbc5-d052-42c7-b701-8fe48fcfd308 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:37.872878 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1096/2217] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:36:37 2022] POST /identity/v3/projects => generated 373 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:36:37.877309 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2da31-9185-45c6-a769-2c266bf7c746 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:37.930353 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1122/2218] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:36:37 2022] PUT /identity/v3/projects/13971c73036e42dc801a51a5b70971b6/users/a9ded5a7330346b8b5492268654d828f/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:37.934903 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6dbbc5-d052-42c7-b701-8fe48fcfd308 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:38.138998 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:38.155973 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6dbbc5-d052-42c7-b701-8fe48fcfd308 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:38.164406 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553385f5-31a7-4091-af98-e40fccc2b7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:38.165139 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553385f5-31a7-4091-af98-e40fccc2b7d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:38.165424 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553385f5-31a7-4091-af98-e40fccc2b7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:38.165863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553385f5-31a7-4091-af98-e40fccc2b7d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:38.167330 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553385f5-31a7-4091-af98-e40fccc2b7d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:38.170687 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553385f5-31a7-4091-af98-e40fccc2b7d8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:38.196398 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1123/2219] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:36:38.196653 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553385f5-31a7-4091-af98-e40fccc2b7d8 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:38.198260 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2da31-9185-45c6-a769-2c266bf7c746 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:38.219039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553385f5-31a7-4091-af98-e40fccc2b7d8 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:38.225866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38636ade-b85b-40db-9443-630423c5f9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:38.227841 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38636ade-b85b-40db-9443-630423c5f9c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:38.228130 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38636ade-b85b-40db-9443-630423c5f9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:38.228517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38636ade-b85b-40db-9443-630423c5f9c6 admin admin] PATH_INFO: `/v3/projects/13971c73036e42dc801a51a5b70971b6/users/a9ded5a7330346b8b5492268654d828f/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:38.230524 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c99a6e-a18f-4e86-9554-6b3b78eb87b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:38.230889 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38636ade-b85b-40db-9443-630423c5f9c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13971c73036e42dc801a51a5b70971b6, user_id=a9ded5a7330346b8b5492268654d828f, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:38.231612 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c99a6e-a18f-4e86-9554-6b3b78eb87b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:38.232099 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c99a6e-a18f-4e86-9554-6b3b78eb87b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:38.232580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c99a6e-a18f-4e86-9554-6b3b78eb87b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:38.234614 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38636ade-b85b-40db-9443-630423c5f9c6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:38.234988 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c99a6e-a18f-4e86-9554-6b3b78eb87b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:38.238657 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c99a6e-a18f-4e86-9554-6b3b78eb87b3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:38.253800 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1124/2220] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:36:38 2022] PUT /identity/v3/projects/13971c73036e42dc801a51a5b70971b6/users/a9ded5a7330346b8b5492268654d828f/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:38.341195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1097/2221] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:36:37 2022] POST /identity/v3/users => generated 287 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:36:38.343682 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38636ade-b85b-40db-9443-630423c5f9c6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:38.478109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38636ade-b85b-40db-9443-630423c5f9c6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:38.483899 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a7c26-1e51-477b-9bf1-e18f6e0ce0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:38.485018 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a7c26-1e51-477b-9bf1-e18f6e0ce0dd admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:38.485527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a7c26-1e51-477b-9bf1-e18f6e0ce0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:38.486021 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a7c26-1e51-477b-9bf1-e18f6e0ce0dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:38.487021 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a7c26-1e51-477b-9bf1-e18f6e0ce0dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:38.490651 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a7c26-1e51-477b-9bf1-e18f6e0ce0dd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:38.512286 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1125/2222] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:36:38.512785 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c99a6e-a18f-4e86-9554-6b3b78eb87b3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:38.534992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c99a6e-a18f-4e86-9554-6b3b78eb87b3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:38.560185 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe8510-97b7-4709-bf0b-5e71827ef8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:38.560908 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe8510-97b7-4709-bf0b-5e71827ef8b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:38.561199 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe8510-97b7-4709-bf0b-5e71827ef8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:38.561770 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe8510-97b7-4709-bf0b-5e71827ef8b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:38.562691 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe8510-97b7-4709-bf0b-5e71827ef8b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:38.565414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe8510-97b7-4709-bf0b-5e71827ef8b2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:38.584669 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a7c26-1e51-477b-9bf1-e18f6e0ce0dd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:38.592970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1098/2223] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:36:38.596648 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a7c26-1e51-477b-9bf1-e18f6e0ce0dd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:38.624094 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726558a-9260-4bd8-b212-a071190fc534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:38.625013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726558a-9260-4bd8-b212-a071190fc534 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:38.625297 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726558a-9260-4bd8-b212-a071190fc534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:38.625784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726558a-9260-4bd8-b212-a071190fc534 admin admin] PATH_INFO: `/v3/projects/e358193434ad40f8b80f402b87c7a3d8/users/42365ff8ea4443159d289d73a9487a72/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:38.668904 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726558a-9260-4bd8-b212-a071190fc534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e358193434ad40f8b80f402b87c7a3d8, user_id=42365ff8ea4443159d289d73a9487a72, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:38.673355 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726558a-9260-4bd8-b212-a071190fc534 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:38.843533 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1126/2224] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:36:38 2022] PUT /identity/v3/projects/e358193434ad40f8b80f402b87c7a3d8/users/42365ff8ea4443159d289d73a9487a72/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:38.855277 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108965e1-4ec6-4c1b-b949-c5758808819a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:38.855940 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108965e1-4ec6-4c1b-b949-c5758808819a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:38.861691 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108965e1-4ec6-4c1b-b949-c5758808819a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:38.950114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-108965e1-4ec6-4c1b-b949-c5758808819a None None] MFA Rules not processed for user `42365ff8ea4443159d289d73a9487a72`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:38.966663 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-108965e1-4ec6-4c1b-b949-c5758808819a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:39.248048 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1099/2225] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:39.260005 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726558a-9260-4bd8-b212-a071190fc534 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:39.261862 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7726558a-9260-4bd8-b212-a071190fc534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:39.393158 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726558a-9260-4bd8-b212-a071190fc534 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:39.400802 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:39.412622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fbf4e1-50c4-44fb-975e-0a3a18885cbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:39.413627 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbf4e1-50c4-44fb-975e-0a3a18885cbe service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:39.414240 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbf4e1-50c4-44fb-975e-0a3a18885cbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:39.414675 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbf4e1-50c4-44fb-975e-0a3a18885cbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:39.432360 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fbf4e1-50c4-44fb-975e-0a3a18885cbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:39.434263 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fbf4e1-50c4-44fb-975e-0a3a18885cbe service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:39.454864 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1127/2226] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:36:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:36:39.527172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108965e1-4ec6-4c1b-b949-c5758808819a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:39.682222 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108965e1-4ec6-4c1b-b949-c5758808819a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:39.696440 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905c3d9-3aad-4cc6-bf44-27c273cb2c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:39.697157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905c3d9-3aad-4cc6-bf44-27c273cb2c7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:39.697445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905c3d9-3aad-4cc6-bf44-27c273cb2c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:39.697791 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905c3d9-3aad-4cc6-bf44-27c273cb2c7c admin admin] PATH_INFO: `/v3/users/42365ff8ea4443159d289d73a9487a72` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:39.702594 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4905c3d9-3aad-4cc6-bf44-27c273cb2c7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=42365ff8ea4443159d289d73a9487a72)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:39.707441 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4905c3d9-3aad-4cc6-bf44-27c273cb2c7c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:39.854001 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:40.020088 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4905c3d9-3aad-4cc6-bf44-27c273cb2c7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42365ff8ea4443159d289d73a9487a72'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:40.026615 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4905c3d9-3aad-4cc6-bf44-27c273cb2c7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42365ff8ea4443159d289d73a9487a72'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:40.178547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4905c3d9-3aad-4cc6-bf44-27c273cb2c7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42365ff8ea4443159d289d73a9487a72'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:40.193776 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4905c3d9-3aad-4cc6-bf44-27c273cb2c7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42365ff8ea4443159d289d73a9487a72'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:40.207724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-4905c3d9-3aad-4cc6-bf44-27c273cb2c7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42365ff8ea4443159d289d73a9487a72'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:40.289789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1100/2227] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:36:39 2022] DELETE /identity/v3/users/42365ff8ea4443159d289d73a9487a72 => generated 0 bytes in 763 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:40.295168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78389d7-3f08-43e1-93da-eb85fd1f8395 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:40.295457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78389d7-3f08-43e1-93da-eb85fd1f8395 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:40.295813 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78389d7-3f08-43e1-93da-eb85fd1f8395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:40.334311 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b78389d7-3f08-43e1-93da-eb85fd1f8395 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:40.335837 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b78389d7-3f08-43e1-93da-eb85fd1f8395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:40.490349 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1128/2228] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:40.497825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905c3d9-3aad-4cc6-bf44-27c273cb2c7c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:40.500208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4905c3d9-3aad-4cc6-bf44-27c273cb2c7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:40.648543 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905c3d9-3aad-4cc6-bf44-27c273cb2c7c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:40.652648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84f6ce7-0879-4599-83d7-27b386a66215 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:40.653168 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f6ce7-0879-4599-83d7-27b386a66215 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:40.654198 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f6ce7-0879-4599-83d7-27b386a66215 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:40.654198 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f6ce7-0879-4599-83d7-27b386a66215 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:40.672785 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84f6ce7-0879-4599-83d7-27b386a66215 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:40.674594 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84f6ce7-0879-4599-83d7-27b386a66215 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:40.699752 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1101/2229] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:36:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:36:41.030264 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:41.630214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78389d7-3f08-43e1-93da-eb85fd1f8395 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:41.631895 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b78389d7-3f08-43e1-93da-eb85fd1f8395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:41.951997 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84f6ce7-0879-4599-83d7-27b386a66215 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:41.957537 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c84f6ce7-0879-4599-83d7-27b386a66215 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:41.964738 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78389d7-3f08-43e1-93da-eb85fd1f8395 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:41.981136 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea977ce6-de19-4a2a-9f83-1ad974646cc0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a932a9390fdd47f3adcf053a60250944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:41.981753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea977ce6-de19-4a2a-9f83-1ad974646cc0 service placement] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:41.982045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea977ce6-de19-4a2a-9f83-1ad974646cc0 service placement] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:41.982326 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea977ce6-de19-4a2a-9f83-1ad974646cc0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:41.985112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ea977ce6-de19-4a2a-9f83-1ad974646cc0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:41.988271 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84f6ce7-0879-4599-83d7-27b386a66215 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:42.037487 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea977ce6-de19-4a2a-9f83-1ad974646cc0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:42.041298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301ea83-99fb-4fa0-b9f6-f7f89ef7082d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:42.052203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301ea83-99fb-4fa0-b9f6-f7f89ef7082d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:42.052993 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301ea83-99fb-4fa0-b9f6-f7f89ef7082d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:42.053770 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301ea83-99fb-4fa0-b9f6-f7f89ef7082d admin admin] PATH_INFO: `/v3/projects/e358193434ad40f8b80f402b87c7a3d8` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:42.054354 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea977ce6-de19-4a2a-9f83-1ad974646cc0 service placement] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:42.059137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301ea83-99fb-4fa0-b9f6-f7f89ef7082d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e358193434ad40f8b80f402b87c7a3d8)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:42.078183 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301ea83-99fb-4fa0-b9f6-f7f89ef7082d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:42.105995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1129/2230] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:36:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:36:42.171991 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:42.188555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba06415-10da-4982-bf4a-4ff33aad4541 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:42.188555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba06415-10da-4982-bf4a-4ff33aad4541 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:42.188555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba06415-10da-4982-bf4a-4ff33aad4541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:42.284213 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2ba06415-10da-4982-bf4a-4ff33aad4541 None None] MFA Rules not processed for user `a9ded5a7330346b8b5492268654d828f`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:42.285474 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2ba06415-10da-4982-bf4a-4ff33aad4541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:42.433412 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d301ea83-99fb-4fa0-b9f6-f7f89ef7082d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e358193434ad40f8b80f402b87c7a3d8', 'request_id': 'req-d301ea83-99fb-4fa0-b9f6-f7f89ef7082d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:42.450245 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d301ea83-99fb-4fa0-b9f6-f7f89ef7082d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e358193434ad40f8b80f402b87c7a3d8', 'request_id': 'req-d301ea83-99fb-4fa0-b9f6-f7f89ef7082d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:42.462913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1130/2231] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:42 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:36:42.470881 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6c157-aa5c-45f6-9d34-dfc9de3d0821 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:42.471434 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6c157-aa5c-45f6-9d34-dfc9de3d0821 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:42.471774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6c157-aa5c-45f6-9d34-dfc9de3d0821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:42.491127 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d301ea83-99fb-4fa0-b9f6-f7f89ef7082d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e358193434ad40f8b80f402b87c7a3d8', 'request_id': 'req-d301ea83-99fb-4fa0-b9f6-f7f89ef7082d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:42.520126 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-4ee6c157-aa5c-45f6-9d34-dfc9de3d0821 None None] MFA Rules not processed for user `23f851892cfa41d08c39d4929267946f`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:42.524412 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4ee6c157-aa5c-45f6-9d34-dfc9de3d0821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:42.586451 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d301ea83-99fb-4fa0-b9f6-f7f89ef7082d admin admin] The token cache is being invalidate because project e358193434ad40f8b80f402b87c7a3d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:36:42.586886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d301ea83-99fb-4fa0-b9f6-f7f89ef7082d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:42.587880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d301ea83-99fb-4fa0-b9f6-f7f89ef7082d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:36:42.589887 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1102/2232] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:36:41 2022] DELETE /identity/v3/projects/e358193434ad40f8b80f402b87c7a3d8 => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:36:42.590832 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301ea83-99fb-4fa0-b9f6-f7f89ef7082d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:42.606567 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d301ea83-99fb-4fa0-b9f6-f7f89ef7082d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:42.773218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1131/2233] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:42 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:42.774505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6e11e-317e-46a8-89d0-9501608a0880 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:42.774820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6e11e-317e-46a8-89d0-9501608a0880 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:42.780485 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c6e11e-317e-46a8-89d0-9501608a0880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:42.835307 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301ea83-99fb-4fa0-b9f6-f7f89ef7082d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:42.847438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64a215-e764-446c-8522-5ff64347952e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:42.848212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64a215-e764-446c-8522-5ff64347952e service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:42.848495 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64a215-e764-446c-8522-5ff64347952e service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:42.849644 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64a215-e764-446c-8522-5ff64347952e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:42.853118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d5c6e11e-317e-46a8-89d0-9501608a0880 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:42.854179 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d5c6e11e-317e-46a8-89d0-9501608a0880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:42.856970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-db64a215-e764-446c-8522-5ff64347952e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:42.991699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1132/2234] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:43.002542 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0699065-a923-4083-b7bf-70de2d8f7744 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:43.002976 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0699065-a923-4083-b7bf-70de2d8f7744 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:43.003583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0699065-a923-4083-b7bf-70de2d8f7744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:43.028911 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d0699065-a923-4083-b7bf-70de2d8f7744 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:43.031592 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d0699065-a923-4083-b7bf-70de2d8f7744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:43.045420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1133/2235] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:43.046573 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0699065-a923-4083-b7bf-70de2d8f7744 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:43.066910 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db64a215-e764-446c-8522-5ff64347952e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:43.076020 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0699065-a923-4083-b7bf-70de2d8f7744 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:43.077279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db64a215-e764-446c-8522-5ff64347952e service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:43.095360 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9534ff3-1d2b-4154-8550-eafedf196569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:43.096140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9534ff3-1d2b-4154-8550-eafedf196569 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:43.096546 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9534ff3-1d2b-4154-8550-eafedf196569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:43.096980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9534ff3-1d2b-4154-8550-eafedf196569 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:43.097911 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9534ff3-1d2b-4154-8550-eafedf196569 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:36:43.098397 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9534ff3-1d2b-4154-8550-eafedf196569 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:43.100964 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9534ff3-1d2b-4154-8550-eafedf196569 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:43.102766 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1103/2236] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:36:42 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:36:43.125397 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1134/2237] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:36:43.126642 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64a215-e764-446c-8522-5ff64347952e service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:43.140531 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64a215-e764-446c-8522-5ff64347952e service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:43.148046 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518c441-6404-4129-8352-18c86d9683cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:43.149676 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518c441-6404-4129-8352-18c86d9683cc admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:43.150287 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518c441-6404-4129-8352-18c86d9683cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:43.150738 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518c441-6404-4129-8352-18c86d9683cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:43.152266 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518c441-6404-4129-8352-18c86d9683cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:43.155913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518c441-6404-4129-8352-18c86d9683cc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:43.169359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:43.388238 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1104/2238] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:36:43 2022] POST /identity/v3/projects => generated 373 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:36:43.392659 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9534ff3-1d2b-4154-8550-eafedf196569 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:43.420190 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9534ff3-1d2b-4154-8550-eafedf196569 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:43.530559 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31563d8-a115-43ee-af8f-8ab8ef950575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:43.531161 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31563d8-a115-43ee-af8f-8ab8ef950575 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:43.531444 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31563d8-a115-43ee-af8f-8ab8ef950575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:43.531908 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31563d8-a115-43ee-af8f-8ab8ef950575 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:43.532778 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31563d8-a115-43ee-af8f-8ab8ef950575 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:43.534877 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31563d8-a115-43ee-af8f-8ab8ef950575 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:43.720406 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1135/2239] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:36:43 2022] POST /identity/v3/users => generated 287 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:36:43.726586 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518c441-6404-4129-8352-18c86d9683cc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:43.765395 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518c441-6404-4129-8352-18c86d9683cc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:43.788178 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6a7d3-a503-468d-96ba-e24b2c7beea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:43.788888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6a7d3-a503-468d-96ba-e24b2c7beea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:43.789172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6a7d3-a503-468d-96ba-e24b2c7beea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:43.789451 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6a7d3-a503-468d-96ba-e24b2c7beea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:43.790208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6a7d3-a503-468d-96ba-e24b2c7beea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:43.791901 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6a7d3-a503-468d-96ba-e24b2c7beea7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:43.817204 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1105/2240] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:36:43.824001 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31563d8-a115-43ee-af8f-8ab8ef950575 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:43.843183 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6a7d3-a503-468d-96ba-e24b2c7beea7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:43.850787 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31563d8-a115-43ee-af8f-8ab8ef950575 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:43.852966 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b5c6a7d3-a503-468d-96ba-e24b2c7beea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:43.856890 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876a620-7048-460d-8bde-4d79fafb5cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:43.857263 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876a620-7048-460d-8bde-4d79fafb5cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:43.857441 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876a620-7048-460d-8bde-4d79fafb5cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:43.857772 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876a620-7048-460d-8bde-4d79fafb5cc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:43.858463 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9876a620-7048-460d-8bde-4d79fafb5cc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:43.860386 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9876a620-7048-460d-8bde-4d79fafb5cc4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:43.875319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1136/2241] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:36:43 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:36:43.882078 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876a620-7048-460d-8bde-4d79fafb5cc4 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:43.904647 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876a620-7048-460d-8bde-4d79fafb5cc4 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:43.916418 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e353e7-d71a-4181-9efe-ae366bd8a10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:43.917929 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e353e7-d71a-4181-9efe-ae366bd8a10b admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:43.919014 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e353e7-d71a-4181-9efe-ae366bd8a10b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:43.919014 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e353e7-d71a-4181-9efe-ae366bd8a10b admin admin] PATH_INFO: `/v3/projects/62ec9078457f461db05068580817ab69/users/936a4ad02f06446f90bea6d95eb17b70/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:44.002012 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e353e7-d71a-4181-9efe-ae366bd8a10b admin admin] RBAC: Authorizing `identity:create_grant(project_id=62ec9078457f461db05068580817ab69, user_id=936a4ad02f06446f90bea6d95eb17b70, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:44.012276 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e353e7-d71a-4181-9efe-ae366bd8a10b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:44.150379 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1137/2242] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:36:43 2022] PUT /identity/v3/projects/62ec9078457f461db05068580817ab69/users/936a4ad02f06446f90bea6d95eb17b70/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:36:44.168230 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f57ce-b276-4f52-b58b-03b1f8dcce55 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:44.174350 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f57ce-b276-4f52-b58b-03b1f8dcce55 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:44.174847 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03f57ce-b276-4f52-b58b-03b1f8dcce55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:44.200761 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6a7d3-a503-468d-96ba-e24b2c7beea7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:44.256025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f03f57ce-b276-4f52-b58b-03b1f8dcce55 None None] MFA Rules not processed for user `936a4ad02f06446f90bea6d95eb17b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:44.287421 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f03f57ce-b276-4f52-b58b-03b1f8dcce55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:44.479984 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8eac32-ab2b-496b-ace5-896f22a811cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:44.482351 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8eac32-ab2b-496b-ace5-896f22a811cd service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:44.482870 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8eac32-ab2b-496b-ace5-896f22a811cd service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:44.483755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8eac32-ab2b-496b-ace5-896f22a811cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:44.488177 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1f8eac32-ab2b-496b-ace5-896f22a811cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:44.625663 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1138/2243] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:36:44.641667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f57ce-b276-4f52-b58b-03b1f8dcce55 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:44.650908 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f03f57ce-b276-4f52-b58b-03b1f8dcce55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:44.753689 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:44.800519 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8eac32-ab2b-496b-ace5-896f22a811cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:44.808593 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8eac32-ab2b-496b-ace5-896f22a811cd service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:44.878736 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03f57ce-b276-4f52-b58b-03b1f8dcce55 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:44.879829 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1106/2244] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:36:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 1027 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:36:44.879972 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51929728-790c-4e2e-9563-640aed185e09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:44.879972 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51929728-790c-4e2e-9563-640aed185e09 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:44.879972 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51929728-790c-4e2e-9563-640aed185e09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:44.879972 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51929728-790c-4e2e-9563-640aed185e09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:44.913097 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51929728-790c-4e2e-9563-640aed185e09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:44.918890 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51929728-790c-4e2e-9563-640aed185e09 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:44.957191 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1139/2245] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:36:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:36:45.572236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8eac32-ab2b-496b-ace5-896f22a811cd service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:36:45.624022 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8eac32-ab2b-496b-ace5-896f22a811cd service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:36:45.641373 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162de5f-07dc-41f8-bec7-9ef5c0922f1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:36:45.642451 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162de5f-07dc-41f8-bec7-9ef5c0922f1f service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:45.642958 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162de5f-07dc-41f8-bec7-9ef5c0922f1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:45.643436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162de5f-07dc-41f8-bec7-9ef5c0922f1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:45.654911 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3162de5f-07dc-41f8-bec7-9ef5c0922f1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:45.808106 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162de5f-07dc-41f8-bec7-9ef5c0922f1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:36:45.809956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162de5f-07dc-41f8-bec7-9ef5c0922f1f service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:36:45.824761 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1107/2246] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:36:45 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:36:46.143694 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:46.832536 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:47.653999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee67df-ae23-40f5-b14b-96099dd5e39a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:47.654283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee67df-ae23-40f5-b14b-96099dd5e39a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:47.654562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee67df-ae23-40f5-b14b-96099dd5e39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:47.704985 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ecee67df-ae23-40f5-b14b-96099dd5e39a None None] MFA Rules not processed for user `936a4ad02f06446f90bea6d95eb17b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:47.708134 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ecee67df-ae23-40f5-b14b-96099dd5e39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:47.741714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1140/2247] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:48.946455 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:49.398570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:50.228204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:50.860256 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04d0ca-7369-4f8a-a1ac-f394425feef0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:36:50.860855 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04d0ca-7369-4f8a-a1ac-f394425feef0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:36:50.861416 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04d0ca-7369-4f8a-a1ac-f394425feef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:36:50.931920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2c04d0ca-7369-4f8a-a1ac-f394425feef0 None None] MFA Rules not processed for user `b1a5d200007a4474a2a25d2ae0915c14`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:36:50.936868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2c04d0ca-7369-4f8a-a1ac-f394425feef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:36:50.954904 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1108/2248] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:36:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:36:51.681797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:52.193439 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:54.647039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:55.219302 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:55.477390 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:56.945840 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:36:57.642096 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:37:06.824837 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=40 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:37:07.292251 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c538f6d-41bb-481d-8053-84aeeaebb9bd None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:07.292542 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c538f6d-41bb-481d-8053-84aeeaebb9bd None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:07.292826 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c538f6d-41bb-481d-8053-84aeeaebb9bd None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:07.294246 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1141/2249] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 30 16:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:37:07.300415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c50633-7413-485f-b1a1-abc836bae88c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:07.300908 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c50633-7413-485f-b1a1-abc836bae88c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:07.301003 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c50633-7413-485f-b1a1-abc836bae88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:07.338857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d1c50633-7413-485f-b1a1-abc836bae88c None None] MFA Rules not processed for user `71cca3708386459197abf6c67505bc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:37:07.340100 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d1c50633-7413-485f-b1a1-abc836bae88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:07.350698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1109/2250] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 30 16:37:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:37:07.362805 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c538f6d-41bb-481d-8053-84aeeaebb9bd None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:07.364678 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9c538f6d-41bb-481d-8053-84aeeaebb9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:07.446623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c538f6d-41bb-481d-8053-84aeeaebb9bd None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:07.450151 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8e306-1843-48d3-b09b-c12d308c6226 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec41c84c9375437590c42621ca5b400a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:07.450791 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8e306-1843-48d3-b09b-c12d308c6226 service glance] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:07.451070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8e306-1843-48d3-b09b-c12d308c6226 service glance] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:07.451350 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8e306-1843-48d3-b09b-c12d308c6226 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:07.462745 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8e306-1843-48d3-b09b-c12d308c6226 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:07.464234 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8e306-1843-48d3-b09b-c12d308c6226 service glance] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:07.475179 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1142/2251] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:37:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:37:07.760275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effedb8c-d414-45e5-b99a-b70c0ee1b028 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:07.760487 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effedb8c-d414-45e5-b99a-b70c0ee1b028 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:07.760759 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effedb8c-d414-45e5-b99a-b70c0ee1b028 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:07.762280 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1110/2252] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 30 16:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:37:07.767277 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7db19d-76bb-42c0-ae8f-d6931a66a25d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:07.767604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7db19d-76bb-42c0-ae8f-d6931a66a25d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:07.767888 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7db19d-76bb-42c0-ae8f-d6931a66a25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:07.787498 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7d7db19d-76bb-42c0-ae8f-d6931a66a25d None None] MFA Rules not processed for user `9b508dd8b7244d5f979e1ce7787d2c67`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:37:07.788797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7d7db19d-76bb-42c0-ae8f-d6931a66a25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:07.841401 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1143/2253] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 30 16:37:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:37:07.853269 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effedb8c-d414-45e5-b99a-b70c0ee1b028 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:07.854970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-effedb8c-d414-45e5-b99a-b70c0ee1b028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:07.869488 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effedb8c-d414-45e5-b99a-b70c0ee1b028 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:07.873013 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd869f2-6d69-4f8a-87a7-e2fb94b5322b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:07.873653 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd869f2-6d69-4f8a-87a7-e2fb94b5322b service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:07.874022 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd869f2-6d69-4f8a-87a7-e2fb94b5322b service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:07.874190 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd869f2-6d69-4f8a-87a7-e2fb94b5322b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:07.882022 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd869f2-6d69-4f8a-87a7-e2fb94b5322b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:07.884700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd869f2-6d69-4f8a-87a7-e2fb94b5322b service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:07.889215 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:37:07.893328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1111/2254] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:37:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:37:07.981971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eece7c8-caa3-4210-8daa-69ff7d956a98 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:07.982563 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eece7c8-caa3-4210-8daa-69ff7d956a98 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:07.982935 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eece7c8-caa3-4210-8daa-69ff7d956a98 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:07.984492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1144/2255] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 30 16:37:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:37:07.990946 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957704a-35a6-4514-8732-868c43c5d2d1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:07.991336 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957704a-35a6-4514-8732-868c43c5d2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:07.992275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5957704a-35a6-4514-8732-868c43c5d2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:08.020300 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5957704a-35a6-4514-8732-868c43c5d2d1 None None] MFA Rules not processed for user `71cca3708386459197abf6c67505bc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:37:08.021426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5957704a-35a6-4514-8732-868c43c5d2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:08.031573 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1112/2256] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 30 16:37:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:37:08.042556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eece7c8-caa3-4210-8daa-69ff7d956a98 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:08.044414 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5eece7c8-caa3-4210-8daa-69ff7d956a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:08.059041 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eece7c8-caa3-4210-8daa-69ff7d956a98 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:08.062707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d64512-729f-4cb8-abd9-8fc9d5f43bb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:08.063345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d64512-729f-4cb8-abd9-8fc9d5f43bb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:08.063702 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d64512-729f-4cb8-abd9-8fc9d5f43bb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:08.064032 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d64512-729f-4cb8-abd9-8fc9d5f43bb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:08.074977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d64512-729f-4cb8-abd9-8fc9d5f43bb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:08.076494 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d64512-729f-4cb8-abd9-8fc9d5f43bb7 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:08.087709 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1145/2257] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:37:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:37:13.992043 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe46f31-a2a5-4815-9190-0d1df547186c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:13.992527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe46f31-a2a5-4815-9190-0d1df547186c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:13.992647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe46f31-a2a5-4815-9190-0d1df547186c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:14.010807 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bbe46f31-a2a5-4815-9190-0d1df547186c None None] MFA Rules not processed for user `d49604bffa3d4e89a631eefa6d4e4f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:37:14.012046 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bbe46f31-a2a5-4815-9190-0d1df547186c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:14.074775 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1113/2258] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 30 16:37:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:37:14.088910 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d64512-729f-4cb8-abd9-8fc9d5f43bb7 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:14.090489 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-29d64512-729f-4cb8-abd9-8fc9d5f43bb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:14.164770 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d64512-729f-4cb8-abd9-8fc9d5f43bb7 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:14.170705 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461608c-420c-482e-ab63-0f2737f6c033 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988b917eea4b48ffae32e23b8555a5c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:14.174662 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461608c-420c-482e-ab63-0f2737f6c033 service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:14.174662 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461608c-420c-482e-ab63-0f2737f6c033 service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:14.174662 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461608c-420c-482e-ab63-0f2737f6c033 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:14.180774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461608c-420c-482e-ab63-0f2737f6c033 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:14.182391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461608c-420c-482e-ab63-0f2737f6c033 service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:14.191070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1146/2259] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 30 16:37:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 16:37:15.521924 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4c060-e466-46a0-aff5-c6f24c843003 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:15.522496 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4c060-e466-46a0-aff5-c6f24c843003 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:15.522924 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4c060-e466-46a0-aff5-c6f24c843003 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:15.525285 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1114/2260] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 30 16:37:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:37:15.532819 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d38947-5ba9-4372-b939-32ae96e3b2c8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:15.533504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d38947-5ba9-4372-b939-32ae96e3b2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:15.533504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d38947-5ba9-4372-b939-32ae96e3b2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:15.558539 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:37:15.585474 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d4d38947-5ba9-4372-b939-32ae96e3b2c8 None None] MFA Rules not processed for user `a932a9390fdd47f3adcf053a60250944`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:37:15.588243 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d4d38947-5ba9-4372-b939-32ae96e3b2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:15.709297 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1147/2261] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 30 16:37:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:37:15.743994 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4c060-e466-46a0-aff5-c6f24c843003 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:15.745941 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-06e4c060-e466-46a0-aff5-c6f24c843003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:15.785004 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4c060-e466-46a0-aff5-c6f24c843003 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:15.791664 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785fed7-f2e3-4ec6-8bf2-6d00a9c11efc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a932a9390fdd47f3adcf053a60250944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:15.792340 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785fed7-f2e3-4ec6-8bf2-6d00a9c11efc service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:15.792676 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785fed7-f2e3-4ec6-8bf2-6d00a9c11efc service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:15.792957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e785fed7-f2e3-4ec6-8bf2-6d00a9c11efc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:15.815383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785fed7-f2e3-4ec6-8bf2-6d00a9c11efc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:15.817225 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e785fed7-f2e3-4ec6-8bf2-6d00a9c11efc service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:15.828412 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1115/2262] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:37:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:37:16.794971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d38947-5ba9-4372-b939-32ae96e3b2c8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:16.798079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d4d38947-5ba9-4372-b939-32ae96e3b2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:16.932902 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d38947-5ba9-4372-b939-32ae96e3b2c8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:16.938480 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaba1cd-9dd4-43fb-a9cc-075add7aaa1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:16.939536 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaba1cd-9dd4-43fb-a9cc-075add7aaa1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:16.939918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaba1cd-9dd4-43fb-a9cc-075add7aaa1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:16.940399 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaba1cd-9dd4-43fb-a9cc-075add7aaa1d admin admin] PATH_INFO: `/v3/users/6b485c044e984d90a76ee17abe5c3eda` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:16.941927 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaba1cd-9dd4-43fb-a9cc-075add7aaa1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b485c044e984d90a76ee17abe5c3eda)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:16.944315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaba1cd-9dd4-43fb-a9cc-075add7aaa1d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:17.152801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cdaba1cd-9dd4-43fb-a9cc-075add7aaa1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b485c044e984d90a76ee17abe5c3eda'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:17.153525 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cdaba1cd-9dd4-43fb-a9cc-075add7aaa1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b485c044e984d90a76ee17abe5c3eda'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:17.160661 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cdaba1cd-9dd4-43fb-a9cc-075add7aaa1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b485c044e984d90a76ee17abe5c3eda'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:17.234440 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cdaba1cd-9dd4-43fb-a9cc-075add7aaa1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b485c044e984d90a76ee17abe5c3eda'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:17.235275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cdaba1cd-9dd4-43fb-a9cc-075add7aaa1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b485c044e984d90a76ee17abe5c3eda'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:17.256287 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1148/2263] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:37:16 2022] DELETE /identity/v3/users/6b485c044e984d90a76ee17abe5c3eda => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:37:17.558244 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785fed7-f2e3-4ec6-8bf2-6d00a9c11efc service placement] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:17.569545 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e785fed7-f2e3-4ec6-8bf2-6d00a9c11efc service placement] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:17.656917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba79f78-3763-43a5-a677-66ba5f5ef8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:17.657698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba79f78-3763-43a5-a677-66ba5f5ef8c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:17.657978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba79f78-3763-43a5-a677-66ba5f5ef8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:17.658251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba79f78-3763-43a5-a677-66ba5f5ef8c8 admin admin] PATH_INFO: `/v3/projects/1933c5b6c74d4a53a6782f833c617b2f` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:17.659501 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba79f78-3763-43a5-a677-66ba5f5ef8c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1933c5b6c74d4a53a6782f833c617b2f)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:17.661421 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba79f78-3763-43a5-a677-66ba5f5ef8c8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:17.824927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7ba79f78-3763-43a5-a677-66ba5f5ef8c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1933c5b6c74d4a53a6782f833c617b2f', 'request_id': 'req-7ba79f78-3763-43a5-a677-66ba5f5ef8c8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:18.079810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7ba79f78-3763-43a5-a677-66ba5f5ef8c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1933c5b6c74d4a53a6782f833c617b2f', 'request_id': 'req-7ba79f78-3763-43a5-a677-66ba5f5ef8c8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:18.079810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7ba79f78-3763-43a5-a677-66ba5f5ef8c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1933c5b6c74d4a53a6782f833c617b2f', 'request_id': 'req-7ba79f78-3763-43a5-a677-66ba5f5ef8c8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:18.084108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7ba79f78-3763-43a5-a677-66ba5f5ef8c8 admin admin] The token cache is being invalidate because project 1933c5b6c74d4a53a6782f833c617b2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:37:18.084399 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7ba79f78-3763-43a5-a677-66ba5f5ef8c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:18.085035 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7ba79f78-3763-43a5-a677-66ba5f5ef8c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:18.086590 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1116/2264] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:37:17 2022] DELETE /identity/v3/projects/1933c5b6c74d4a53a6782f833c617b2f => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:37:18.090699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaba1cd-9dd4-43fb-a9cc-075add7aaa1d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:18.092410 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cdaba1cd-9dd4-43fb-a9cc-075add7aaa1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:18.106431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaba1cd-9dd4-43fb-a9cc-075add7aaa1d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:18.116521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:37:18.194835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3406c6c6-5f89-494a-bb5d-9cba02b945ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:18.195783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406c6c6-5f89-494a-bb5d-9cba02b945ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:18.196066 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406c6c6-5f89-494a-bb5d-9cba02b945ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:18.196341 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406c6c6-5f89-494a-bb5d-9cba02b945ed admin admin] PATH_INFO: `/v3/users/a9ded5a7330346b8b5492268654d828f` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:18.199323 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406c6c6-5f89-494a-bb5d-9cba02b945ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9ded5a7330346b8b5492268654d828f)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:18.202158 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3406c6c6-5f89-494a-bb5d-9cba02b945ed admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:18.494896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3406c6c6-5f89-494a-bb5d-9cba02b945ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9ded5a7330346b8b5492268654d828f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:18.495689 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3406c6c6-5f89-494a-bb5d-9cba02b945ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9ded5a7330346b8b5492268654d828f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:18.503736 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3406c6c6-5f89-494a-bb5d-9cba02b945ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9ded5a7330346b8b5492268654d828f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:18.574671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3406c6c6-5f89-494a-bb5d-9cba02b945ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9ded5a7330346b8b5492268654d828f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:18.576232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3406c6c6-5f89-494a-bb5d-9cba02b945ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9ded5a7330346b8b5492268654d828f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:18.598397 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1149/2265] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:37:18 2022] DELETE /identity/v3/users/a9ded5a7330346b8b5492268654d828f => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:37:18.923737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba79f78-3763-43a5-a677-66ba5f5ef8c8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:18.923737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7ba79f78-3763-43a5-a677-66ba5f5ef8c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:18.950664 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba79f78-3763-43a5-a677-66ba5f5ef8c8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:18.981189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:37:19.058633 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467d723-6384-421e-95eb-d8345d96283a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:19.059577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467d723-6384-421e-95eb-d8345d96283a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:19.059864 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467d723-6384-421e-95eb-d8345d96283a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:19.060138 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467d723-6384-421e-95eb-d8345d96283a admin admin] PATH_INFO: `/v3/projects/13971c73036e42dc801a51a5b70971b6` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:19.061416 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467d723-6384-421e-95eb-d8345d96283a admin admin] RBAC: Authorizing `identity:delete_project(project_id=13971c73036e42dc801a51a5b70971b6)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:19.063347 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467d723-6384-421e-95eb-d8345d96283a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:19.210681 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b467d723-6384-421e-95eb-d8345d96283a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13971c73036e42dc801a51a5b70971b6', 'request_id': 'req-b467d723-6384-421e-95eb-d8345d96283a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:19.218622 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b467d723-6384-421e-95eb-d8345d96283a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13971c73036e42dc801a51a5b70971b6', 'request_id': 'req-b467d723-6384-421e-95eb-d8345d96283a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:19.225880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b467d723-6384-421e-95eb-d8345d96283a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13971c73036e42dc801a51a5b70971b6', 'request_id': 'req-b467d723-6384-421e-95eb-d8345d96283a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:19.312933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b467d723-6384-421e-95eb-d8345d96283a admin admin] The token cache is being invalidate because project 13971c73036e42dc801a51a5b70971b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:37:19.316356 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b467d723-6384-421e-95eb-d8345d96283a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:19.316356 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b467d723-6384-421e-95eb-d8345d96283a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:19.316356 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1117/2266] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:37:18 2022] DELETE /identity/v3/projects/13971c73036e42dc801a51a5b70971b6 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:37:19.320949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843983e-e85f-47d1-be5a-2346f7fd76ce None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:19.321478 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843983e-e85f-47d1-be5a-2346f7fd76ce None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:19.321615 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843983e-e85f-47d1-be5a-2346f7fd76ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:19.362986 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a843983e-e85f-47d1-be5a-2346f7fd76ce None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:37:19.364269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a843983e-e85f-47d1-be5a-2346f7fd76ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:19.451912 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1150/2267] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:37:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:37:19.523825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865b2ac-ec3b-40ef-8baf-cb688a4c86be None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:19.524182 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865b2ac-ec3b-40ef-8baf-cb688a4c86be None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:19.524378 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9865b2ac-ec3b-40ef-8baf-cb688a4c86be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:19.566801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9865b2ac-ec3b-40ef-8baf-cb688a4c86be None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:37:19.567758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9865b2ac-ec3b-40ef-8baf-cb688a4c86be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:19.578841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1118/2268] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:37:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:37:19.583448 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843983e-e85f-47d1-be5a-2346f7fd76ce None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:19.593965 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843983e-e85f-47d1-be5a-2346f7fd76ce None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:19.599063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa58bb-4d26-467b-9962-98815bb73f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:19.599793 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa58bb-4d26-467b-9962-98815bb73f75 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:19.600234 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa58bb-4d26-467b-9962-98815bb73f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:19.600532 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa58bb-4d26-467b-9962-98815bb73f75 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:19.603863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa58bb-4d26-467b-9962-98815bb73f75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:37:19.604520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa58bb-4d26-467b-9962-98815bb73f75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:19.606461 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aa58bb-4d26-467b-9962-98815bb73f75 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:19.636580 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1151/2269] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:37:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:37:19.643063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865b2ac-ec3b-40ef-8baf-cb688a4c86be None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:19.656694 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9865b2ac-ec3b-40ef-8baf-cb688a4c86be None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:19.662376 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f498dd-d6a5-4827-903a-33361a81efd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:19.663160 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f498dd-d6a5-4827-903a-33361a81efd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:19.663576 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f498dd-d6a5-4827-903a-33361a81efd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:19.663917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f498dd-d6a5-4827-903a-33361a81efd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:19.664721 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f498dd-d6a5-4827-903a-33361a81efd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:19.667138 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f498dd-d6a5-4827-903a-33361a81efd8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:19.794366 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1119/2270] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:37:19 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:37:19.795628 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa58bb-4d26-467b-9962-98815bb73f75 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:19.810741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa58bb-4d26-467b-9962-98815bb73f75 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:19.857611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:37:19.942848 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ab18a-9512-434c-ae47-df5eb0876e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:19.943949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ab18a-9512-434c-ae47-df5eb0876e2d admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:19.948983 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ab18a-9512-434c-ae47-df5eb0876e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:19.948983 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ab18a-9512-434c-ae47-df5eb0876e2d admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:19.948983 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ab18a-9512-434c-ae47-df5eb0876e2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:19.954599 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ab18a-9512-434c-ae47-df5eb0876e2d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:20.078800 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1152/2271] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:37:19 2022] POST /identity/v3/users => generated 293 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:37:20.083123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f498dd-d6a5-4827-903a-33361a81efd8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:20.094325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f498dd-d6a5-4827-903a-33361a81efd8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:20.099170 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50030d8f-3b2d-453c-b46c-bf72117aa831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:20.099900 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50030d8f-3b2d-453c-b46c-bf72117aa831 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:20.100188 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50030d8f-3b2d-453c-b46c-bf72117aa831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:20.100476 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50030d8f-3b2d-453c-b46c-bf72117aa831 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:20.101195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50030d8f-3b2d-453c-b46c-bf72117aa831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:20.102815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50030d8f-3b2d-453c-b46c-bf72117aa831 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:20.117507 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1120/2272] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:37:20.119212 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ab18a-9512-434c-ae47-df5eb0876e2d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:20.129608 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ab18a-9512-434c-ae47-df5eb0876e2d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:20.134328 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab4b08-8e9a-424a-89f0-95cda33eb203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:20.138320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab4b08-8e9a-424a-89f0-95cda33eb203 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:20.138320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab4b08-8e9a-424a-89f0-95cda33eb203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:20.138320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab4b08-8e9a-424a-89f0-95cda33eb203 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:20.138320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab4b08-8e9a-424a-89f0-95cda33eb203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:20.138734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab4b08-8e9a-424a-89f0-95cda33eb203 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:20.151838 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1153/2273] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:37:20.157003 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50030d8f-3b2d-453c-b46c-bf72117aa831 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:20.170522 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50030d8f-3b2d-453c-b46c-bf72117aa831 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:20.175994 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81fcc99-6df7-4599-85dc-4380b15d606a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:20.176895 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fcc99-6df7-4599-85dc-4380b15d606a admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:20.177182 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fcc99-6df7-4599-85dc-4380b15d606a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:20.177461 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81fcc99-6df7-4599-85dc-4380b15d606a admin admin] PATH_INFO: `/v3/projects/47af405ee8224cc197974a153137c4f0/users/5a95e9088846422c940942a82081c647/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:20.196730 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81fcc99-6df7-4599-85dc-4380b15d606a admin admin] RBAC: Authorizing `identity:create_grant(project_id=47af405ee8224cc197974a153137c4f0, user_id=5a95e9088846422c940942a82081c647, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:20.200315 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81fcc99-6df7-4599-85dc-4380b15d606a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:20.273831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1121/2274] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:37:20 2022] PUT /identity/v3/projects/47af405ee8224cc197974a153137c4f0/users/5a95e9088846422c940942a82081c647/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:37:20.284636 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209605d-7d60-4c69-bbed-2442862136d5 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:20.284930 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209605d-7d60-4c69-bbed-2442862136d5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:20.285215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209605d-7d60-4c69-bbed-2442862136d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:20.317335 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e209605d-7d60-4c69-bbed-2442862136d5 None None] MFA Rules not processed for user `5a95e9088846422c940942a82081c647`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:37:20.318605 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e209605d-7d60-4c69-bbed-2442862136d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:20.401760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1154/2275] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:37:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:37:20.412962 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81fcc99-6df7-4599-85dc-4380b15d606a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:20.415004 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d81fcc99-6df7-4599-85dc-4380b15d606a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:20.493076 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:37:20.512375 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81fcc99-6df7-4599-85dc-4380b15d606a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:20.517560 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919a70e-e2ca-4f9b-9975-74533e94c116 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:20.518685 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919a70e-e2ca-4f9b-9975-74533e94c116 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:20.555069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919a70e-e2ca-4f9b-9975-74533e94c116 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:20.555069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919a70e-e2ca-4f9b-9975-74533e94c116 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:20.555069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919a70e-e2ca-4f9b-9975-74533e94c116 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:20.555069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919a70e-e2ca-4f9b-9975-74533e94c116 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:20.557640 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1122/2276] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:37:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:37:20.581222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e209605d-7d60-4c69-bbed-2442862136d5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:20.655712 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e209605d-7d60-4c69-bbed-2442862136d5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:20.666818 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd3979-8494-43bb-95ee-fa84f05920c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:20.667768 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd3979-8494-43bb-95ee-fa84f05920c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:20.668121 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd3979-8494-43bb-95ee-fa84f05920c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:20.668470 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd3979-8494-43bb-95ee-fa84f05920c0 admin admin] PATH_INFO: `/v3/users/5a95e9088846422c940942a82081c647` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:20.669788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd3979-8494-43bb-95ee-fa84f05920c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a95e9088846422c940942a82081c647)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:20.671911 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dd3979-8494-43bb-95ee-fa84f05920c0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:21.252160 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:37:21.390750 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d4dd3979-8494-43bb-95ee-fa84f05920c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a95e9088846422c940942a82081c647'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:21.391499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d4dd3979-8494-43bb-95ee-fa84f05920c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a95e9088846422c940942a82081c647'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:21.398996 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d4dd3979-8494-43bb-95ee-fa84f05920c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a95e9088846422c940942a82081c647'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:21.484693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d4dd3979-8494-43bb-95ee-fa84f05920c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a95e9088846422c940942a82081c647'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:21.485949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d4dd3979-8494-43bb-95ee-fa84f05920c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a95e9088846422c940942a82081c647'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:21.529792 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1155/2277] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:37:20 2022] DELETE /identity/v3/users/5a95e9088846422c940942a82081c647 => generated 0 bytes in 949 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:37:21.534173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71211ce5-ac4c-424b-9be3-35d63050764d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:21.535054 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71211ce5-ac4c-424b-9be3-35d63050764d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:21.535339 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71211ce5-ac4c-424b-9be3-35d63050764d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:21.555660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-71211ce5-ac4c-424b-9be3-35d63050764d None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:37:21.557706 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-71211ce5-ac4c-424b-9be3-35d63050764d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:21.628465 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1123/2278] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:37:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:37:21.639794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd3979-8494-43bb-95ee-fa84f05920c0 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:21.641520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d4dd3979-8494-43bb-95ee-fa84f05920c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:21.734837 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd3979-8494-43bb-95ee-fa84f05920c0 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:21.738796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a7bd5-cf92-4088-ae13-2b13675d0d20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:21.739241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a7bd5-cf92-4088-ae13-2b13675d0d20 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:21.739561 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a7bd5-cf92-4088-ae13-2b13675d0d20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:21.739842 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a7bd5-cf92-4088-ae13-2b13675d0d20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:21.750403 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a7bd5-cf92-4088-ae13-2b13675d0d20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:21.751934 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3a7bd5-cf92-4088-ae13-2b13675d0d20 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:21.764895 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1156/2279] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:37:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:37:22.361615 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71211ce5-ac4c-424b-9be3-35d63050764d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:22.363230 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-71211ce5-ac4c-424b-9be3-35d63050764d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:22.395086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71211ce5-ac4c-424b-9be3-35d63050764d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:22.401528 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035d14af-528d-457d-966a-ad15e36c1dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:22.404404 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d14af-528d-457d-966a-ad15e36c1dd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:22.404404 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d14af-528d-457d-966a-ad15e36c1dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:22.404404 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d14af-528d-457d-966a-ad15e36c1dd1 admin admin] PATH_INFO: `/v3/projects/47af405ee8224cc197974a153137c4f0` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:22.404817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d14af-528d-457d-966a-ad15e36c1dd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47af405ee8224cc197974a153137c4f0)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:22.407035 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035d14af-528d-457d-966a-ad15e36c1dd1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:22.590098 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-035d14af-528d-457d-966a-ad15e36c1dd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47af405ee8224cc197974a153137c4f0', 'request_id': 'req-035d14af-528d-457d-966a-ad15e36c1dd1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:22.597297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-035d14af-528d-457d-966a-ad15e36c1dd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47af405ee8224cc197974a153137c4f0', 'request_id': 'req-035d14af-528d-457d-966a-ad15e36c1dd1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:22.604148 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-035d14af-528d-457d-966a-ad15e36c1dd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47af405ee8224cc197974a153137c4f0', 'request_id': 'req-035d14af-528d-457d-966a-ad15e36c1dd1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:22.689738 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-035d14af-528d-457d-966a-ad15e36c1dd1 admin admin] The token cache is being invalidate because project 47af405ee8224cc197974a153137c4f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:37:22.690086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-035d14af-528d-457d-966a-ad15e36c1dd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:22.690683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-035d14af-528d-457d-966a-ad15e36c1dd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:22.692326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1124/2280] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:37:22 2022] DELETE /identity/v3/projects/47af405ee8224cc197974a153137c4f0 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:37:22.705945 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936caf7-05bb-46d3-88ca-135c592493c3 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:22.706532 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936caf7-05bb-46d3-88ca-135c592493c3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:22.706532 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936caf7-05bb-46d3-88ca-135c592493c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:22.728802 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7936caf7-05bb-46d3-88ca-135c592493c3 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:37:22.730055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7936caf7-05bb-46d3-88ca-135c592493c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:22.832367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1157/2281] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:37:22.842043 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bb661-bdd4-4e9c-b606-25666906eb41 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:22.842464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bb661-bdd4-4e9c-b606-25666906eb41 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:22.842859 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bb661-bdd4-4e9c-b606-25666906eb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:22.878554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-242bb661-bdd4-4e9c-b606-25666906eb41 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:37:22.880722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-242bb661-bdd4-4e9c-b606-25666906eb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:22.902032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1125/2282] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:37:22.906819 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936caf7-05bb-46d3-88ca-135c592493c3 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:22.918277 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936caf7-05bb-46d3-88ca-135c592493c3 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:22.923803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb723746-914f-4021-a0f1-8d56c2af4aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:22.924493 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb723746-914f-4021-a0f1-8d56c2af4aeb admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:22.924813 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb723746-914f-4021-a0f1-8d56c2af4aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:22.925056 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb723746-914f-4021-a0f1-8d56c2af4aeb admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:22.925820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb723746-914f-4021-a0f1-8d56c2af4aeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:37:22.926112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb723746-914f-4021-a0f1-8d56c2af4aeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:22.927788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb723746-914f-4021-a0f1-8d56c2af4aeb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:22.946723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1158/2283] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:37:22.948503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242bb661-bdd4-4e9c-b606-25666906eb41 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:22.960754 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242bb661-bdd4-4e9c-b606-25666906eb41 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:22.965567 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced2f66-eb46-43f0-8c68-34c43e2b5503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:22.966231 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced2f66-eb46-43f0-8c68-34c43e2b5503 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:22.966512 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced2f66-eb46-43f0-8c68-34c43e2b5503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:22.966789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced2f66-eb46-43f0-8c68-34c43e2b5503 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:22.967542 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced2f66-eb46-43f0-8c68-34c43e2b5503 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:22.969541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced2f66-eb46-43f0-8c68-34c43e2b5503 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:23.122495 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1126/2284] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:37:22 2022] POST /identity/v3/projects => generated 379 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:37:23.126566 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb723746-914f-4021-a0f1-8d56c2af4aeb admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:23.139214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb723746-914f-4021-a0f1-8d56c2af4aeb admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:23.181860 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:37:23.214387 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97acba75-5a09-4e39-9937-91e86b742a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:23.215371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97acba75-5a09-4e39-9937-91e86b742a2c admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:23.215869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97acba75-5a09-4e39-9937-91e86b742a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:23.216311 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97acba75-5a09-4e39-9937-91e86b742a2c admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:23.217357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97acba75-5a09-4e39-9937-91e86b742a2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:23.220022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97acba75-5a09-4e39-9937-91e86b742a2c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:23.314775 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1159/2285] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:37:23 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:37:23.319325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced2f66-eb46-43f0-8c68-34c43e2b5503 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:23.336270 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced2f66-eb46-43f0-8c68-34c43e2b5503 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:23.343988 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411c07a-0a94-4b41-96a0-5200c9f597b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:23.345534 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411c07a-0a94-4b41-96a0-5200c9f597b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:23.345534 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411c07a-0a94-4b41-96a0-5200c9f597b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:23.345534 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411c07a-0a94-4b41-96a0-5200c9f597b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:23.357061 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411c07a-0a94-4b41-96a0-5200c9f597b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:23.359851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411c07a-0a94-4b41-96a0-5200c9f597b7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:23.374612 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1127/2286] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:37:23 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:37:23.378828 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97acba75-5a09-4e39-9937-91e86b742a2c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:23.398432 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97acba75-5a09-4e39-9937-91e86b742a2c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:23.406009 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe2096-be80-4c36-8fca-eea7449bfcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:23.407553 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe2096-be80-4c36-8fca-eea7449bfcf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:23.408083 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe2096-be80-4c36-8fca-eea7449bfcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:23.408370 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe2096-be80-4c36-8fca-eea7449bfcf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:23.409230 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe2096-be80-4c36-8fca-eea7449bfcf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:23.411015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe2096-be80-4c36-8fca-eea7449bfcf9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:23.424829 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1160/2287] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:37:23 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:37:23.428879 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411c07a-0a94-4b41-96a0-5200c9f597b7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:23.453111 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411c07a-0a94-4b41-96a0-5200c9f597b7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:23.458573 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58a623-8f25-48d0-8dd7-e52265a6d9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:23.459617 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58a623-8f25-48d0-8dd7-e52265a6d9f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:23.459970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58a623-8f25-48d0-8dd7-e52265a6d9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:23.460317 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58a623-8f25-48d0-8dd7-e52265a6d9f1 admin admin] PATH_INFO: `/v3/projects/99847b649bfd4ca6a33cb2ea97ad0469/users/bb8a8a4e8b634322a92992379c98b721/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:23.488771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58a623-8f25-48d0-8dd7-e52265a6d9f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99847b649bfd4ca6a33cb2ea97ad0469, user_id=bb8a8a4e8b634322a92992379c98b721, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:23.499895 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58a623-8f25-48d0-8dd7-e52265a6d9f1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:23.586022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb11ee-4170-4be3-bb02-d722aa8cf177 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:23.586308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb11ee-4170-4be3-bb02-d722aa8cf177 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:23.586479 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1128/2288] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:37:23 2022] PUT /identity/v3/projects/99847b649bfd4ca6a33cb2ea97ad0469/users/bb8a8a4e8b634322a92992379c98b721/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:37:23.586587 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53eb11ee-4170-4be3-bb02-d722aa8cf177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:23.610282 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-53eb11ee-4170-4be3-bb02-d722aa8cf177 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:37:23.611600 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-53eb11ee-4170-4be3-bb02-d722aa8cf177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:23.695914 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1161/2289] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:37:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:37:23.716318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58a623-8f25-48d0-8dd7-e52265a6d9f1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:23.718990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ed58a623-8f25-48d0-8dd7-e52265a6d9f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:23.726208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:37:23.795321 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58a623-8f25-48d0-8dd7-e52265a6d9f1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:23.801659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea0459-4926-475d-b2ab-b9f7a11f724b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:23.802606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea0459-4926-475d-b2ab-b9f7a11f724b service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:23.802606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea0459-4926-475d-b2ab-b9f7a11f724b service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:23.802797 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea0459-4926-475d-b2ab-b9f7a11f724b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:23.813129 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea0459-4926-475d-b2ab-b9f7a11f724b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:23.814913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea0459-4926-475d-b2ab-b9f7a11f724b service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:23.826828 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1129/2290] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:37:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:37:30.448892 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91274d51-4618-4b30-94ac-3b38522f6eef None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:30.449471 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91274d51-4618-4b30-94ac-3b38522f6eef None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:30.449471 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91274d51-4618-4b30-94ac-3b38522f6eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:30.479427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-91274d51-4618-4b30-94ac-3b38522f6eef None None] MFA Rules not processed for user `bb8a8a4e8b634322a92992379c98b721`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:37:30.480893 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-91274d51-4618-4b30-94ac-3b38522f6eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:30.561032 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1162/2291] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:37:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:37:30.565484 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea0459-4926-475d-b2ab-b9f7a11f724b service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:30.577664 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea0459-4926-475d-b2ab-b9f7a11f724b service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:30.585713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87270b45-b913-4aa7-a85f-70c9b881f6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:30.587141 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87270b45-b913-4aa7-a85f-70c9b881f6f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:30.587641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87270b45-b913-4aa7-a85f-70c9b881f6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:30.588010 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87270b45-b913-4aa7-a85f-70c9b881f6f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:30.589062 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87270b45-b913-4aa7-a85f-70c9b881f6f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:30.591143 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87270b45-b913-4aa7-a85f-70c9b881f6f6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:30.711542 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1130/2292] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:37:30 2022] POST /identity/v3/projects => generated 377 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:37:30.715344 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91274d51-4618-4b30-94ac-3b38522f6eef None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:30.734816 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91274d51-4618-4b30-94ac-3b38522f6eef None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:30.830030 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:37:30.881094 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60f5f4-b824-4015-9b33-162229ccc25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:30.882050 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60f5f4-b824-4015-9b33-162229ccc25a admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:30.882571 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60f5f4-b824-4015-9b33-162229ccc25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:30.883001 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa60f5f4-b824-4015-9b33-162229ccc25a admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:30.883964 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa60f5f4-b824-4015-9b33-162229ccc25a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:30.886862 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa60f5f4-b824-4015-9b33-162229ccc25a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:31.069977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1163/2293] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:37:30 2022] POST /identity/v3/users => generated 295 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:37:31.075497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87270b45-b913-4aa7-a85f-70c9b881f6f6 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:31.089577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87270b45-b913-4aa7-a85f-70c9b881f6f6 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:31.098632 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6603530-d836-4175-8e36-c936fc329944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:31.109808 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6603530-d836-4175-8e36-c936fc329944 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:31.110460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6603530-d836-4175-8e36-c936fc329944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:31.110797 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6603530-d836-4175-8e36-c936fc329944 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:31.111604 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6603530-d836-4175-8e36-c936fc329944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:31.116549 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6603530-d836-4175-8e36-c936fc329944 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:31.139858 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1131/2294] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:37:31.143677 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60f5f4-b824-4015-9b33-162229ccc25a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:31.154328 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa60f5f4-b824-4015-9b33-162229ccc25a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:31.159877 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2334e-7fcd-47d5-905c-f0fa1bc966a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:31.161089 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2334e-7fcd-47d5-905c-f0fa1bc966a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:31.161426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2334e-7fcd-47d5-905c-f0fa1bc966a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:31.161805 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2334e-7fcd-47d5-905c-f0fa1bc966a0 admin admin] PATH_INFO: `/v3/projects/3d71b79e0b9841fc8392ca2fcb24b55f/users/f6aae1ef404f4511a8bd234cff0eac5b/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:31.203309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2334e-7fcd-47d5-905c-f0fa1bc966a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d71b79e0b9841fc8392ca2fcb24b55f, user_id=f6aae1ef404f4511a8bd234cff0eac5b, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:31.205972 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2334e-7fcd-47d5-905c-f0fa1bc966a0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:31.326289 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1164/2295] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:37:31 2022] PUT /identity/v3/projects/3d71b79e0b9841fc8392ca2fcb24b55f/users/f6aae1ef404f4511a8bd234cff0eac5b/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:37:31.329892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6603530-d836-4175-8e36-c936fc329944 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:31.343012 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6603530-d836-4175-8e36-c936fc329944 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:31.407439 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e64cc-a649-467c-b45c-793fc8dfa836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:31.408409 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e64cc-a649-467c-b45c-793fc8dfa836 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:31.408699 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e64cc-a649-467c-b45c-793fc8dfa836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:31.408977 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e64cc-a649-467c-b45c-793fc8dfa836 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:31.409702 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e64cc-a649-467c-b45c-793fc8dfa836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:31.411549 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e64cc-a649-467c-b45c-793fc8dfa836 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:31.423023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1132/2296] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:37:31.427576 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2334e-7fcd-47d5-905c-f0fa1bc966a0 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:31.440063 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2334e-7fcd-47d5-905c-f0fa1bc966a0 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:31.445210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e793a1c-44a8-42b4-be16-e0220752db11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:31.445629 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e793a1c-44a8-42b4-be16-e0220752db11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:31.445908 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e793a1c-44a8-42b4-be16-e0220752db11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:31.446180 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e793a1c-44a8-42b4-be16-e0220752db11 admin admin] PATH_INFO: `/v3/projects/3d71b79e0b9841fc8392ca2fcb24b55f/users/f6aae1ef404f4511a8bd234cff0eac5b/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:31.449042 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e793a1c-44a8-42b4-be16-e0220752db11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d71b79e0b9841fc8392ca2fcb24b55f, user_id=f6aae1ef404f4511a8bd234cff0eac5b, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:31.451754 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e793a1c-44a8-42b4-be16-e0220752db11 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:31.467140 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1165/2297] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:37:31 2022] PUT /identity/v3/projects/3d71b79e0b9841fc8392ca2fcb24b55f/users/f6aae1ef404f4511a8bd234cff0eac5b/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:37:37.122113 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b56f1-6e6c-4598-b27e-068505ec4a4a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:37.123201 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b56f1-6e6c-4598-b27e-068505ec4a4a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:37.128891 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b56f1-6e6c-4598-b27e-068505ec4a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:37.172022 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4f3b56f1-6e6c-4598-b27e-068505ec4a4a None None] MFA Rules not processed for user `d49604bffa3d4e89a631eefa6d4e4f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:37:37.174901 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4f3b56f1-6e6c-4598-b27e-068505ec4a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:37.415660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1133/2298] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 30 16:37:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:37:37.440059 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e793a1c-44a8-42b4-be16-e0220752db11 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:37.442708 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7e793a1c-44a8-42b4-be16-e0220752db11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:37.554357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e793a1c-44a8-42b4-be16-e0220752db11 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:37.565782 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562b104-0501-49b8-8927-dedb60894bbd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988b917eea4b48ffae32e23b8555a5c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:37.567301 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562b104-0501-49b8-8927-dedb60894bbd service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:37.569037 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562b104-0501-49b8-8927-dedb60894bbd service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:37.569341 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562b104-0501-49b8-8927-dedb60894bbd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:37.592887 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562b104-0501-49b8-8927-dedb60894bbd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:37.595310 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562b104-0501-49b8-8927-dedb60894bbd service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:37.629296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1166/2299] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 30 16:37:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 16:37:39.116723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9671f67-8fd8-4261-80d8-37be8bf15242 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:39.117456 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9671f67-8fd8-4261-80d8-37be8bf15242 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:39.117456 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9671f67-8fd8-4261-80d8-37be8bf15242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:39.147653 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e9671f67-8fd8-4261-80d8-37be8bf15242 None None] MFA Rules not processed for user `d49604bffa3d4e89a631eefa6d4e4f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:37:39.149251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e9671f67-8fd8-4261-80d8-37be8bf15242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:39.158432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1134/2300] 10.222.0.67 () {58 vars in 967 bytes} [Thu Jun 30 16:37:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:37:39.177493 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562b104-0501-49b8-8927-dedb60894bbd service swift] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:39.196106 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562b104-0501-49b8-8927-dedb60894bbd service swift] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:39.211004 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad503627-2a21-4c8e-bd9c-473856ee90c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988b917eea4b48ffae32e23b8555a5c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:39.211699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad503627-2a21-4c8e-bd9c-473856ee90c1 service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:39.211981 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad503627-2a21-4c8e-bd9c-473856ee90c1 service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:39.212258 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad503627-2a21-4c8e-bd9c-473856ee90c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:39.251773 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad503627-2a21-4c8e-bd9c-473856ee90c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:39.258813 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad503627-2a21-4c8e-bd9c-473856ee90c1 service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:39.285026 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1167/2301] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 30 16:37:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 16:37:40.350304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37386a-b275-41c6-a07e-88d0cd337221 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:40.350790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37386a-b275-41c6-a07e-88d0cd337221 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:40.351009 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f37386a-b275-41c6-a07e-88d0cd337221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:40.372338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7f37386a-b275-41c6-a07e-88d0cd337221 None None] MFA Rules not processed for user `d49604bffa3d4e89a631eefa6d4e4f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:37:40.373702 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7f37386a-b275-41c6-a07e-88d0cd337221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:40.381516 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1135/2302] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 30 16:37:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:37:40.396844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad503627-2a21-4c8e-bd9c-473856ee90c1 service swift] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:40.411046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad503627-2a21-4c8e-bd9c-473856ee90c1 service swift] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:40.421533 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ed5d8-94a7-4295-ad58-637e43805d5f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988b917eea4b48ffae32e23b8555a5c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:40.422268 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ed5d8-94a7-4295-ad58-637e43805d5f service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:40.422538 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ed5d8-94a7-4295-ad58-637e43805d5f service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:40.422812 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ed5d8-94a7-4295-ad58-637e43805d5f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:40.434791 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ed5d8-94a7-4295-ad58-637e43805d5f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:40.436824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ed5d8-94a7-4295-ad58-637e43805d5f service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:40.451707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1168/2303] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 30 16:37:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 16:37:41.494408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc9ff9-2024-4e59-832c-9bbcfcea472b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:41.494888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc9ff9-2024-4e59-832c-9bbcfcea472b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:41.495006 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc9ff9-2024-4e59-832c-9bbcfcea472b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:41.502020 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:37:41.515607 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-debc9ff9-2024-4e59-832c-9bbcfcea472b None None] MFA Rules not processed for user `d49604bffa3d4e89a631eefa6d4e4f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:37:41.516972 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-debc9ff9-2024-4e59-832c-9bbcfcea472b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:41.525063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1136/2304] 10.222.0.67 () {58 vars in 967 bytes} [Thu Jun 30 16:37:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:37:41.539235 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ed5d8-94a7-4295-ad58-637e43805d5f service swift] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:41.553582 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ed5d8-94a7-4295-ad58-637e43805d5f service swift] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:41.560713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafc21d-749e-4449-bb6e-160a85f3ab07 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988b917eea4b48ffae32e23b8555a5c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:41.561404 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafc21d-749e-4449-bb6e-160a85f3ab07 service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:41.561663 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafc21d-749e-4449-bb6e-160a85f3ab07 service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:41.562058 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafc21d-749e-4449-bb6e-160a85f3ab07 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:41.571853 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafc21d-749e-4449-bb6e-160a85f3ab07 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:41.573556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafc21d-749e-4449-bb6e-160a85f3ab07 service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:41.584617 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1169/2305] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 30 16:37:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 16:37:42.139318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abab7f4-e98a-4253-ac89-ae560457da65 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:42.139318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abab7f4-e98a-4253-ac89-ae560457da65 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:42.139318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abab7f4-e98a-4253-ac89-ae560457da65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:42.178573 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6abab7f4-e98a-4253-ac89-ae560457da65 None None] MFA Rules not processed for user `f6aae1ef404f4511a8bd234cff0eac5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:37:42.180070 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6abab7f4-e98a-4253-ac89-ae560457da65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:42.314493 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1137/2306] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:37:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:37:42.357145 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafc21d-749e-4449-bb6e-160a85f3ab07 service swift] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:42.357145 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-aaafc21d-749e-4449-bb6e-160a85f3ab07 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:42.475520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:37:42.525668 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafc21d-749e-4449-bb6e-160a85f3ab07 service swift] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:42.532955 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b7ff2-b643-4284-9aed-50bad56744a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:42.537697 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b7ff2-b643-4284-9aed-50bad56744a6 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:42.538231 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b7ff2-b643-4284-9aed-50bad56744a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:42.538702 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b7ff2-b643-4284-9aed-50bad56744a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:42.629701 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b7ff2-b643-4284-9aed-50bad56744a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:42.631456 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b7ff2-b643-4284-9aed-50bad56744a6 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:42.641809 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1170/2307] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:37:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:37:44.988238 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abab7f4-e98a-4253-ac89-ae560457da65 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:44.990199 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6abab7f4-e98a-4253-ac89-ae560457da65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:45.166548 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abab7f4-e98a-4253-ac89-ae560457da65 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:45.194334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938364a9-9f88-4b7f-ad1f-653bd6f54242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:45.195261 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938364a9-9f88-4b7f-ad1f-653bd6f54242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:45.195571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938364a9-9f88-4b7f-ad1f-653bd6f54242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:45.195856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938364a9-9f88-4b7f-ad1f-653bd6f54242 admin admin] PATH_INFO: `/v3/users/936a4ad02f06446f90bea6d95eb17b70` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:45.200823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938364a9-9f88-4b7f-ad1f-653bd6f54242 admin admin] RBAC: Authorizing `identity:delete_user(user_id=936a4ad02f06446f90bea6d95eb17b70)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:45.202830 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938364a9-9f88-4b7f-ad1f-653bd6f54242 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:45.520776 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-938364a9-9f88-4b7f-ad1f-653bd6f54242 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '936a4ad02f06446f90bea6d95eb17b70'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:45.521556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-938364a9-9f88-4b7f-ad1f-653bd6f54242 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '936a4ad02f06446f90bea6d95eb17b70'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:45.624058 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-938364a9-9f88-4b7f-ad1f-653bd6f54242 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '936a4ad02f06446f90bea6d95eb17b70'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:45.644812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-938364a9-9f88-4b7f-ad1f-653bd6f54242 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '936a4ad02f06446f90bea6d95eb17b70'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:45.647424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-938364a9-9f88-4b7f-ad1f-653bd6f54242 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '936a4ad02f06446f90bea6d95eb17b70'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:45.684069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:37:45.688872 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1138/2308] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:37:44 2022] DELETE /identity/v3/users/936a4ad02f06446f90bea6d95eb17b70 => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:37:45.693660 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747f5fc-fa10-4d08-b784-cf9a76ea66ec None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:45.696535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747f5fc-fa10-4d08-b784-cf9a76ea66ec None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:45.696535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747f5fc-fa10-4d08-b784-cf9a76ea66ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:45.714635 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c747f5fc-fa10-4d08-b784-cf9a76ea66ec None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:37:45.716238 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c747f5fc-fa10-4d08-b784-cf9a76ea66ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:45.834225 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1171/2309] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:37:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:37:45.840297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938364a9-9f88-4b7f-ad1f-653bd6f54242 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:45.846063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-938364a9-9f88-4b7f-ad1f-653bd6f54242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:45.997257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938364a9-9f88-4b7f-ad1f-653bd6f54242 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:46.007747 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6fdf49-a950-4c65-8174-51b4816bafce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:46.008686 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fdf49-a950-4c65-8174-51b4816bafce service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:46.011345 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fdf49-a950-4c65-8174-51b4816bafce service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:46.011898 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6fdf49-a950-4c65-8174-51b4816bafce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:46.030109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6fdf49-a950-4c65-8174-51b4816bafce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:46.045171 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6fdf49-a950-4c65-8174-51b4816bafce service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:46.045171 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1139/2310] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:37:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:37:46.470327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c747f5fc-fa10-4d08-b784-cf9a76ea66ec None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:46.473411 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c747f5fc-fa10-4d08-b784-cf9a76ea66ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:46.488965 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c747f5fc-fa10-4d08-b784-cf9a76ea66ec None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:46.494495 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efc339-31da-4df3-9bb6-502ebdc27695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:46.496424 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efc339-31da-4df3-9bb6-502ebdc27695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:46.497310 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efc339-31da-4df3-9bb6-502ebdc27695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:46.497993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efc339-31da-4df3-9bb6-502ebdc27695 admin admin] PATH_INFO: `/v3/projects/62ec9078457f461db05068580817ab69` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:46.499678 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efc339-31da-4df3-9bb6-502ebdc27695 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62ec9078457f461db05068580817ab69)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:46.502186 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efc339-31da-4df3-9bb6-502ebdc27695 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:46.661173 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-96efc339-31da-4df3-9bb6-502ebdc27695 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62ec9078457f461db05068580817ab69', 'request_id': 'req-96efc339-31da-4df3-9bb6-502ebdc27695'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:46.748995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-96efc339-31da-4df3-9bb6-502ebdc27695 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62ec9078457f461db05068580817ab69', 'request_id': 'req-96efc339-31da-4df3-9bb6-502ebdc27695'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:46.756787 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-96efc339-31da-4df3-9bb6-502ebdc27695 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62ec9078457f461db05068580817ab69', 'request_id': 'req-96efc339-31da-4df3-9bb6-502ebdc27695'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:46.776308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-96efc339-31da-4df3-9bb6-502ebdc27695 admin admin] The token cache is being invalidate because project 62ec9078457f461db05068580817ab69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:37:46.777348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-96efc339-31da-4df3-9bb6-502ebdc27695 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:46.778706 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-96efc339-31da-4df3-9bb6-502ebdc27695 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:37:46.780774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1172/2311] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:37:46 2022] DELETE /identity/v3/projects/62ec9078457f461db05068580817ab69 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:37:49.682401 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83cf75-3cbc-4cdc-ae08-7cb55496639d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:49.682969 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83cf75-3cbc-4cdc-ae08-7cb55496639d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:49.683283 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b83cf75-3cbc-4cdc-ae08-7cb55496639d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:49.710372 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6b83cf75-3cbc-4cdc-ae08-7cb55496639d None None] MFA Rules not processed for user `96d7af5479ce46b99e333d91128b826e`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:37:49.712328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6b83cf75-3cbc-4cdc-ae08-7cb55496639d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:49.782744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1140/2312] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:37:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:37:50.072987 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b23488-a068-4f90-805f-23ba7d4d734e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:50.073599 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b23488-a068-4f90-805f-23ba7d4d734e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:50.073686 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b23488-a068-4f90-805f-23ba7d4d734e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:50.089009 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:37:50.099152 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-62b23488-a068-4f90-805f-23ba7d4d734e None None] MFA Rules not processed for user `d49604bffa3d4e89a631eefa6d4e4f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:37:50.100607 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-62b23488-a068-4f90-805f-23ba7d4d734e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:50.179751 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1173/2313] 10.222.0.67 () {58 vars in 967 bytes} [Thu Jun 30 16:37:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:37:50.192905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83cf75-3cbc-4cdc-ae08-7cb55496639d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:50.194363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6b83cf75-3cbc-4cdc-ae08-7cb55496639d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:50.324181 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b83cf75-3cbc-4cdc-ae08-7cb55496639d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:50.344447 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb0696f-4efa-459e-9bf4-f30f8ebe5fe4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988b917eea4b48ffae32e23b8555a5c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:50.344972 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0696f-4efa-459e-9bf4-f30f8ebe5fe4 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:50.345260 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0696f-4efa-459e-9bf4-f30f8ebe5fe4 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:50.345541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb0696f-4efa-459e-9bf4-f30f8ebe5fe4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:50.355008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0696f-4efa-459e-9bf4-f30f8ebe5fe4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:50.358285 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb0696f-4efa-459e-9bf4-f30f8ebe5fe4 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:50.367627 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1141/2314] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 30 16:37:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 16:37:56.275424 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b23488-a068-4f90-805f-23ba7d4d734e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:37:56.277806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-62b23488-a068-4f90-805f-23ba7d4d734e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:56.412452 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b23488-a068-4f90-805f-23ba7d4d734e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:37:56.420034 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ed644-9079-43d2-b73b-b5df6448d8cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:37:56.420034 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ed644-9079-43d2-b73b-b5df6448d8cc service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:37:56.420034 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ed644-9079-43d2-b73b-b5df6448d8cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:37:56.420034 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ed644-9079-43d2-b73b-b5df6448d8cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:37:56.422853 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a12ed644-9079-43d2-b73b-b5df6448d8cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:37:56.500107 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ed644-9079-43d2-b73b-b5df6448d8cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:37:56.501619 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12ed644-9079-43d2-b73b-b5df6448d8cc service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:37:56.512800 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1174/2315] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:37:56 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:38:00.363806 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=83 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:38:02.329240 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a49e0-6af6-4571-ace1-2eb602e80ea1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:38:02.329951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a49e0-6af6-4571-ace1-2eb602e80ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:38:02.329951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a49e0-6af6-4571-ace1-2eb602e80ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:38:02.349461 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a91a49e0-6af6-4571-ace1-2eb602e80ea1 None None] MFA Rules not processed for user `0a26028451b345968fe86774f44736dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:38:02.350787 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a91a49e0-6af6-4571-ace1-2eb602e80ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:38:02.436501 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1142/2316] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:38:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:38:06.012875 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79653e-68f0-4ddc-961e-b2830795cdc3 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:38:06.013955 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79653e-68f0-4ddc-961e-b2830795cdc3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:38:06.013955 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79653e-68f0-4ddc-961e-b2830795cdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:38:06.036104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1b79653e-68f0-4ddc-961e-b2830795cdc3 None None] MFA Rules not processed for user `bb8a8a4e8b634322a92992379c98b721`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:38:06.036104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1b79653e-68f0-4ddc-961e-b2830795cdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:38:06.100324 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1175/2317] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:38:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:38:12.424659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=25 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:38:12.813805 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd11e8-67e4-413a-9542-3263da5c1c9c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:38:12.814113 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd11e8-67e4-413a-9542-3263da5c1c9c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:38:12.814590 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd11e8-67e4-413a-9542-3263da5c1c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:38:12.834699 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b2cd11e8-67e4-413a-9542-3263da5c1c9c None None] MFA Rules not processed for user `2de23770f4374decb95ff94f3e7f793b`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:38:12.836406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b2cd11e8-67e4-413a-9542-3263da5c1c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:38:12.924868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1143/2318] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:38:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:38:16.092043 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:38:22.907916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=25 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:38:39.551120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79653e-68f0-4ddc-961e-b2830795cdc3 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:38:39.554918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1b79653e-68f0-4ddc-961e-b2830795cdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:38:39.567170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79653e-68f0-4ddc-961e-b2830795cdc3 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:38:39.571995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea7832-716c-44d0-a7b8-fbb1ae9cea93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:38:39.572658 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea7832-716c-44d0-a7b8-fbb1ae9cea93 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:38:39.573037 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea7832-716c-44d0-a7b8-fbb1ae9cea93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:38:39.573415 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea7832-716c-44d0-a7b8-fbb1ae9cea93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:38:39.575527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-75ea7832-716c-44d0-a7b8-fbb1ae9cea93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:38:39.656470 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea7832-716c-44d0-a7b8-fbb1ae9cea93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:38:39.657943 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea7832-716c-44d0-a7b8-fbb1ae9cea93 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:38:39.669924 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1176/2319] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:38:39 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:38:47.547081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd11e8-67e4-413a-9542-3263da5c1c9c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:38:47.549917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b2cd11e8-67e4-413a-9542-3263da5c1c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:38:47.640858 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd11e8-67e4-413a-9542-3263da5c1c9c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:38:47.645295 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee118b1-b65f-4dbb-8354-32028d9f2ef2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:38:47.645910 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee118b1-b65f-4dbb-8354-32028d9f2ef2 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:38:47.646234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee118b1-b65f-4dbb-8354-32028d9f2ef2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:38:47.646961 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee118b1-b65f-4dbb-8354-32028d9f2ef2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:38:47.648855 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8ee118b1-b65f-4dbb-8354-32028d9f2ef2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:38:47.730374 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee118b1-b65f-4dbb-8354-32028d9f2ef2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:38:47.731788 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee118b1-b65f-4dbb-8354-32028d9f2ef2 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:38:47.746007 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1144/2320] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:38:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:38:49.665122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=30 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:38:50.739301 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea7832-716c-44d0-a7b8-fbb1ae9cea93 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:38:50.750594 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea7832-716c-44d0-a7b8-fbb1ae9cea93 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:38:50.756770 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538904a-a6fa-45ee-ad88-0d1140574f9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:38:50.757430 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538904a-a6fa-45ee-ad88-0d1140574f9e service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:38:50.757711 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538904a-a6fa-45ee-ad88-0d1140574f9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:38:50.757986 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538904a-a6fa-45ee-ad88-0d1140574f9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:38:50.760133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7538904a-a6fa-45ee-ad88-0d1140574f9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:38:50.775067 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7538904a-a6fa-45ee-ad88-0d1140574f9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:38:50.783643 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7538904a-a6fa-45ee-ad88-0d1140574f9e service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:38:50.788784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1177/2321] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:38:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:38:57.739272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=54 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:39:00.783241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=6 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:39:02.743954 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153474fb-768e-418d-ae74-d12749e44d86 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:02.744498 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153474fb-768e-418d-ae74-d12749e44d86 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:02.746220 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153474fb-768e-418d-ae74-d12749e44d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:02.764116 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-153474fb-768e-418d-ae74-d12749e44d86 None None] MFA Rules not processed for user `7261ab46436e4546b8331aa8387cfa1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:39:02.765364 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-153474fb-768e-418d-ae74-d12749e44d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:02.828288 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1145/2322] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:39:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:39:02.964110 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538904a-a6fa-45ee-ad88-0d1140574f9e service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:02.969636 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538904a-a6fa-45ee-ad88-0d1140574f9e service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:02.973398 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd45ad-88ad-48af-8fc5-44fd4b94cab3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:02.974287 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd45ad-88ad-48af-8fc5-44fd4b94cab3 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:02.974661 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd45ad-88ad-48af-8fc5-44fd4b94cab3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:02.974956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd45ad-88ad-48af-8fc5-44fd4b94cab3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:02.976951 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-53cd45ad-88ad-48af-8fc5-44fd4b94cab3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:02.988765 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd45ad-88ad-48af-8fc5-44fd4b94cab3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:02.990726 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd45ad-88ad-48af-8fc5-44fd4b94cab3 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:03.000707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1178/2323] 10.222.0.67 () {60 vars in 1396 bytes} [Thu Jun 30 16:39:02 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3252 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:39:03.217523 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153474fb-768e-418d-ae74-d12749e44d86 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:03.233190 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153474fb-768e-418d-ae74-d12749e44d86 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:03.236811 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312678d-c281-483d-a932-ea10ec4c59c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:03.237509 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312678d-c281-483d-a932-ea10ec4c59c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:03.237785 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312678d-c281-483d-a932-ea10ec4c59c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:03.238060 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312678d-c281-483d-a932-ea10ec4c59c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:03.240646 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9312678d-c281-483d-a932-ea10ec4c59c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:03.250675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312678d-c281-483d-a932-ea10ec4c59c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:03.252136 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312678d-c281-483d-a932-ea10ec4c59c1 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:03.261148 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1146/2324] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:39:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:39:03.746495 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd45ad-88ad-48af-8fc5-44fd4b94cab3 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:03.747979 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-53cd45ad-88ad-48af-8fc5-44fd4b94cab3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:03.832669 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd45ad-88ad-48af-8fc5-44fd4b94cab3 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:03.839142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bee519-fb26-42bf-8ff1-03e108164fdd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a932a9390fdd47f3adcf053a60250944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:03.839919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bee519-fb26-42bf-8ff1-03e108164fdd service placement] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:03.840203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bee519-fb26-42bf-8ff1-03e108164fdd service placement] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:03.840482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bee519-fb26-42bf-8ff1-03e108164fdd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:03.842369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-70bee519-fb26-42bf-8ff1-03e108164fdd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:03.861109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bee519-fb26-42bf-8ff1-03e108164fdd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:03.864694 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bee519-fb26-42bf-8ff1-03e108164fdd service placement] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:03.874881 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1179/2325] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:39:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:39:07.417475 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312678d-c281-483d-a932-ea10ec4c59c1 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:07.427625 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312678d-c281-483d-a932-ea10ec4c59c1 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:07.434364 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbeda1-d202-4355-bc31-156d4f7cec8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:07.435046 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbeda1-d202-4355-bc31-156d4f7cec8f service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:07.435337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbeda1-d202-4355-bc31-156d4f7cec8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:07.435645 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbeda1-d202-4355-bc31-156d4f7cec8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:07.437484 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-54fbeda1-d202-4355-bc31-156d4f7cec8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:07.449157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbeda1-d202-4355-bc31-156d4f7cec8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:07.450614 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbeda1-d202-4355-bc31-156d4f7cec8f service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:07.460688 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1147/2326] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:39:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:39:09.325509 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bee519-fb26-42bf-8ff1-03e108164fdd service placement] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:09.335812 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bee519-fb26-42bf-8ff1-03e108164fdd service placement] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:09.339862 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494a2f0-7857-41a3-9b69-c058cafa5bb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:09.344069 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494a2f0-7857-41a3-9b69-c058cafa5bb6 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:09.344069 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494a2f0-7857-41a3-9b69-c058cafa5bb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:09.344069 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494a2f0-7857-41a3-9b69-c058cafa5bb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:09.344069 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2494a2f0-7857-41a3-9b69-c058cafa5bb6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:09.410892 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494a2f0-7857-41a3-9b69-c058cafa5bb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:09.413555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494a2f0-7857-41a3-9b69-c058cafa5bb6 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:09.425104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1180/2327] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:39:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:39:10.389576 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbeda1-d202-4355-bc31-156d4f7cec8f service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:10.391101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-54fbeda1-d202-4355-bc31-156d4f7cec8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:10.404528 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbeda1-d202-4355-bc31-156d4f7cec8f service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:10.408431 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed70dc8-9752-491f-8af7-5b39f9716e96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:10.414128 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed70dc8-9752-491f-8af7-5b39f9716e96 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:10.414128 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed70dc8-9752-491f-8af7-5b39f9716e96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:10.414128 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed70dc8-9752-491f-8af7-5b39f9716e96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:10.414128 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5ed70dc8-9752-491f-8af7-5b39f9716e96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:10.424716 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed70dc8-9752-491f-8af7-5b39f9716e96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:10.426319 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed70dc8-9752-491f-8af7-5b39f9716e96 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:10.442380 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1148/2328] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:39:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:39:10.507619 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494a2f0-7857-41a3-9b69-c058cafa5bb6 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:10.509748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2494a2f0-7857-41a3-9b69-c058cafa5bb6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:10.597462 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494a2f0-7857-41a3-9b69-c058cafa5bb6 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:10.603200 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041141d-39dd-4916-ac39-a96c2413bbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:10.604264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041141d-39dd-4916-ac39-a96c2413bbd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:10.604744 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041141d-39dd-4916-ac39-a96c2413bbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:10.605024 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3041141d-39dd-4916-ac39-a96c2413bbd7 admin admin] PATH_INFO: `/v3/users/96d7af5479ce46b99e333d91128b826e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:10.606315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3041141d-39dd-4916-ac39-a96c2413bbd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96d7af5479ce46b99e333d91128b826e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:10.608448 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3041141d-39dd-4916-ac39-a96c2413bbd7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:10.627803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:39:10.946185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3041141d-39dd-4916-ac39-a96c2413bbd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96d7af5479ce46b99e333d91128b826e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:10.947013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3041141d-39dd-4916-ac39-a96c2413bbd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96d7af5479ce46b99e333d91128b826e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:10.955704 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3041141d-39dd-4916-ac39-a96c2413bbd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96d7af5479ce46b99e333d91128b826e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:11.085002 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3041141d-39dd-4916-ac39-a96c2413bbd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96d7af5479ce46b99e333d91128b826e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:11.085672 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3041141d-39dd-4916-ac39-a96c2413bbd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96d7af5479ce46b99e333d91128b826e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:11.113219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1181/2329] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:39:10 2022] DELETE /identity/v3/users/96d7af5479ce46b99e333d91128b826e => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:39:11.418844 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed70dc8-9752-491f-8af7-5b39f9716e96 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:11.441894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed70dc8-9752-491f-8af7-5b39f9716e96 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:11.503728 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666db80-94ff-4ad4-b680-271e840d59f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:11.507812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666db80-94ff-4ad4-b680-271e840d59f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:11.507812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666db80-94ff-4ad4-b680-271e840d59f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:11.507812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666db80-94ff-4ad4-b680-271e840d59f3 admin admin] PATH_INFO: `/v3/projects/4df4c3aa7c8044dcbd46064e0e495a84` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:11.507812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666db80-94ff-4ad4-b680-271e840d59f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4df4c3aa7c8044dcbd46064e0e495a84)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:11.509116 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666db80-94ff-4ad4-b680-271e840d59f3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:11.634966 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d666db80-94ff-4ad4-b680-271e840d59f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4df4c3aa7c8044dcbd46064e0e495a84', 'request_id': 'req-d666db80-94ff-4ad4-b680-271e840d59f3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:11.641263 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d666db80-94ff-4ad4-b680-271e840d59f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4df4c3aa7c8044dcbd46064e0e495a84', 'request_id': 'req-d666db80-94ff-4ad4-b680-271e840d59f3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:11.647652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d666db80-94ff-4ad4-b680-271e840d59f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4df4c3aa7c8044dcbd46064e0e495a84', 'request_id': 'req-d666db80-94ff-4ad4-b680-271e840d59f3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:11.651348 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:39:11.718709 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d666db80-94ff-4ad4-b680-271e840d59f3 admin admin] The token cache is being invalidate because project 4df4c3aa7c8044dcbd46064e0e495a84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:39:11.718994 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d666db80-94ff-4ad4-b680-271e840d59f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:11.719673 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d666db80-94ff-4ad4-b680-271e840d59f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:11.721341 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1149/2330] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:39:11 2022] DELETE /identity/v3/projects/4df4c3aa7c8044dcbd46064e0e495a84 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:39:13.578046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041141d-39dd-4916-ac39-a96c2413bbd7 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:13.581378 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3041141d-39dd-4916-ac39-a96c2413bbd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:13.670073 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3041141d-39dd-4916-ac39-a96c2413bbd7 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:13.743349 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ebee6-5065-4891-ad19-2fb5089ffe6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:13.743957 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ebee6-5065-4891-ad19-2fb5089ffe6d service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:13.744270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ebee6-5065-4891-ad19-2fb5089ffe6d service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:13.744526 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2ebee6-5065-4891-ad19-2fb5089ffe6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:13.746483 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9b2ebee6-5065-4891-ad19-2fb5089ffe6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:13.778636 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:39:13.813334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ebee6-5065-4891-ad19-2fb5089ffe6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:13.815013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2ebee6-5065-4891-ad19-2fb5089ffe6d service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:13.827211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1182/2331] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:39:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:39:19.471575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666db80-94ff-4ad4-b680-271e840d59f3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:19.475301 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d666db80-94ff-4ad4-b680-271e840d59f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:19.506749 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666db80-94ff-4ad4-b680-271e840d59f3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:19.522262 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbb7d9-9520-4e89-ad03-a744ac5bd12f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:19.522942 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbb7d9-9520-4e89-ad03-a744ac5bd12f service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:19.523226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbb7d9-9520-4e89-ad03-a744ac5bd12f service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:19.523524 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbb7d9-9520-4e89-ad03-a744ac5bd12f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:19.525417 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-24cbb7d9-9520-4e89-ad03-a744ac5bd12f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:19.543152 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cbb7d9-9520-4e89-ad03-a744ac5bd12f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:19.545421 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cbb7d9-9520-4e89-ad03-a744ac5bd12f service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:19.559900 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1150/2332] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:39:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:39:22.032784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ebee6-5065-4891-ad19-2fb5089ffe6d service nova] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:22.044786 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2ebee6-5065-4891-ad19-2fb5089ffe6d service nova] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:22.050682 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c388bd-3e4c-4425-8878-62f13b249149 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:22.051346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c388bd-3e4c-4425-8878-62f13b249149 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:22.051675 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c388bd-3e4c-4425-8878-62f13b249149 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:22.051951 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c388bd-3e4c-4425-8878-62f13b249149 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:22.054718 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d7c388bd-3e4c-4425-8878-62f13b249149 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:22.069265 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c388bd-3e4c-4425-8878-62f13b249149 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:22.070389 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c388bd-3e4c-4425-8878-62f13b249149 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:22.082939 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1183/2333] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:39:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:39:25.020169 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc605b-4562-4f1b-8413-3fd3267d275a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:25.021125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc605b-4562-4f1b-8413-3fd3267d275a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:25.021125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bc605b-4562-4f1b-8413-3fd3267d275a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:25.039633 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-56bc605b-4562-4f1b-8413-3fd3267d275a None None] MFA Rules not processed for user `18f2856b1d3742a1aca89b4aeec0a3bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:39:25.040794 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-56bc605b-4562-4f1b-8413-3fd3267d275a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:25.098591 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1151/2334] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:39:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:39:25.423222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c388bd-3e4c-4425-8878-62f13b249149 service nova] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:25.424707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d7c388bd-3e4c-4425-8878-62f13b249149 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:25.482481 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c388bd-3e4c-4425-8878-62f13b249149 service nova] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:25.486045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51329621-09a0-4f83-a377-ace1f1b25c39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:25.496856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51329621-09a0-4f83-a377-ace1f1b25c39 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:25.496856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51329621-09a0-4f83-a377-ace1f1b25c39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:25.496856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51329621-09a0-4f83-a377-ace1f1b25c39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:25.496856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-51329621-09a0-4f83-a377-ace1f1b25c39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:25.500498 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51329621-09a0-4f83-a377-ace1f1b25c39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:25.501956 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51329621-09a0-4f83-a377-ace1f1b25c39 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:25.510884 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1184/2335] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:39:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:39:35.091457 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=33 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:39:35.506944 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=46 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:39:38.426899 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc605b-4562-4f1b-8413-3fd3267d275a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:38.429436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-56bc605b-4562-4f1b-8413-3fd3267d275a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:38.441744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bc605b-4562-4f1b-8413-3fd3267d275a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:38.446690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605cfed-0848-49b1-b33a-74755147ffa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:38.447361 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605cfed-0848-49b1-b33a-74755147ffa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:38.447690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605cfed-0848-49b1-b33a-74755147ffa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:38.447975 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605cfed-0848-49b1-b33a-74755147ffa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:38.449895 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4605cfed-0848-49b1-b33a-74755147ffa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:38.514665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605cfed-0848-49b1-b33a-74755147ffa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:38.516164 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605cfed-0848-49b1-b33a-74755147ffa5 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:38.526098 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1152/2336] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:39:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:39:40.913895 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51329621-09a0-4f83-a377-ace1f1b25c39 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:40.915407 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-51329621-09a0-4f83-a377-ace1f1b25c39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:40.933867 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51329621-09a0-4f83-a377-ace1f1b25c39 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:40.940263 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae69a97-1941-4e01-bf29-fbf777891aa1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:40.941180 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae69a97-1941-4e01-bf29-fbf777891aa1 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:40.941580 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae69a97-1941-4e01-bf29-fbf777891aa1 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:40.941867 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae69a97-1941-4e01-bf29-fbf777891aa1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:40.943741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8ae69a97-1941-4e01-bf29-fbf777891aa1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:40.957827 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae69a97-1941-4e01-bf29-fbf777891aa1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:40.959531 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae69a97-1941-4e01-bf29-fbf777891aa1 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:40.972121 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1185/2337] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:39:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:39:42.653029 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605cfed-0848-49b1-b33a-74755147ffa5 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:42.654632 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4605cfed-0848-49b1-b33a-74755147ffa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:42.667742 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605cfed-0848-49b1-b33a-74755147ffa5 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:42.672552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8098d6a7-54ed-4b40-9e06-4eb548734118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:42.673433 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098d6a7-54ed-4b40-9e06-4eb548734118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:42.673725 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098d6a7-54ed-4b40-9e06-4eb548734118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:42.674030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098d6a7-54ed-4b40-9e06-4eb548734118 admin admin] PATH_INFO: `/v3/users/2de23770f4374decb95ff94f3e7f793b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:42.675227 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098d6a7-54ed-4b40-9e06-4eb548734118 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2de23770f4374decb95ff94f3e7f793b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:42.677199 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098d6a7-54ed-4b40-9e06-4eb548734118 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:42.899704 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3662ff-89d9-4a3e-903c-1697556303b7 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:42.900195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3662ff-89d9-4a3e-903c-1697556303b7 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:42.900314 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3662ff-89d9-4a3e-903c-1697556303b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:42.920283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-9d3662ff-89d9-4a3e-903c-1697556303b7 None None] MFA Rules not processed for user `0a26028451b345968fe86774f44736dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:39:42.921590 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-9d3662ff-89d9-4a3e-903c-1697556303b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:42.925058 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8098d6a7-54ed-4b40-9e06-4eb548734118 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2de23770f4374decb95ff94f3e7f793b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:42.925764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8098d6a7-54ed-4b40-9e06-4eb548734118 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2de23770f4374decb95ff94f3e7f793b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:42.930615 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1186/2338] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:39:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:39:43.000867 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8098d6a7-54ed-4b40-9e06-4eb548734118 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2de23770f4374decb95ff94f3e7f793b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:43.022887 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8098d6a7-54ed-4b40-9e06-4eb548734118 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2de23770f4374decb95ff94f3e7f793b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:43.023744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8098d6a7-54ed-4b40-9e06-4eb548734118 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2de23770f4374decb95ff94f3e7f793b'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:43.046586 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1153/2339] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:39:42 2022] DELETE /identity/v3/users/2de23770f4374decb95ff94f3e7f793b => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:39:43.362575 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3662ff-89d9-4a3e-903c-1697556303b7 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:43.378201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3662ff-89d9-4a3e-903c-1697556303b7 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:43.442062 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f3659f-96af-4e9d-8c7d-039bbbb3d720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:43.443025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3659f-96af-4e9d-8c7d-039bbbb3d720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:43.443311 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3659f-96af-4e9d-8c7d-039bbbb3d720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:43.443775 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f3659f-96af-4e9d-8c7d-039bbbb3d720 admin admin] PATH_INFO: `/v3/projects/839a189179c543c6a704a968170ec022` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:43.445131 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f3659f-96af-4e9d-8c7d-039bbbb3d720 admin admin] RBAC: Authorizing `identity:delete_project(project_id=839a189179c543c6a704a968170ec022)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:43.447111 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f3659f-96af-4e9d-8c7d-039bbbb3d720 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:43.563532 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-94f3659f-96af-4e9d-8c7d-039bbbb3d720 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '839a189179c543c6a704a968170ec022', 'request_id': 'req-94f3659f-96af-4e9d-8c7d-039bbbb3d720'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:43.640302 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-94f3659f-96af-4e9d-8c7d-039bbbb3d720 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '839a189179c543c6a704a968170ec022', 'request_id': 'req-94f3659f-96af-4e9d-8c7d-039bbbb3d720'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:43.647910 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-94f3659f-96af-4e9d-8c7d-039bbbb3d720 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '839a189179c543c6a704a968170ec022', 'request_id': 'req-94f3659f-96af-4e9d-8c7d-039bbbb3d720'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:43.658371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-94f3659f-96af-4e9d-8c7d-039bbbb3d720 admin admin] The token cache is being invalidate because project 839a189179c543c6a704a968170ec022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:39:43.658670 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-94f3659f-96af-4e9d-8c7d-039bbbb3d720 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:43.659306 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-94f3659f-96af-4e9d-8c7d-039bbbb3d720 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:43.663147 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1187/2340] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:39:43 2022] DELETE /identity/v3/projects/839a189179c543c6a704a968170ec022 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:39:43.666551 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093c3c2-c03a-4934-9ac4-0af496a49fce None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:43.667104 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093c3c2-c03a-4934-9ac4-0af496a49fce None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:43.667232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093c3c2-c03a-4934-9ac4-0af496a49fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:43.688671 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e093c3c2-c03a-4934-9ac4-0af496a49fce None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:39:43.689896 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e093c3c2-c03a-4934-9ac4-0af496a49fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:43.766685 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1154/2341] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:39:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:39:43.775589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc514d-6ac2-4347-8a99-00bb5a1ab9ad None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:43.776008 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc514d-6ac2-4347-8a99-00bb5a1ab9ad None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:43.776353 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc514d-6ac2-4347-8a99-00bb5a1ab9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:43.801735 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d4dc514d-6ac2-4347-8a99-00bb5a1ab9ad None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:39:43.807176 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d4dc514d-6ac2-4347-8a99-00bb5a1ab9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:43.812678 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1188/2342] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:39:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:39:43.816589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e093c3c2-c03a-4934-9ac4-0af496a49fce None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:43.827500 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e093c3c2-c03a-4934-9ac4-0af496a49fce None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:43.832782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08bf811-c03d-4d25-9090-788d4244825c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:43.833450 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bf811-c03d-4d25-9090-788d4244825c admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:43.833732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bf811-c03d-4d25-9090-788d4244825c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:43.834007 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08bf811-c03d-4d25-9090-788d4244825c admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:43.837595 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08bf811-c03d-4d25-9090-788d4244825c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:39:43.837595 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08bf811-c03d-4d25-9090-788d4244825c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:43.837595 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08bf811-c03d-4d25-9090-788d4244825c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:43.839207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:39:43.854960 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1155/2343] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:39:43.856681 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc514d-6ac2-4347-8a99-00bb5a1ab9ad None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:43.872947 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc514d-6ac2-4347-8a99-00bb5a1ab9ad None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:43.872947 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89cf0ce-a43e-40a5-b15f-2bcce1d4526d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:43.873825 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89cf0ce-a43e-40a5-b15f-2bcce1d4526d admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:43.874133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89cf0ce-a43e-40a5-b15f-2bcce1d4526d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:43.874408 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89cf0ce-a43e-40a5-b15f-2bcce1d4526d admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:43.875088 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89cf0ce-a43e-40a5-b15f-2bcce1d4526d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:43.877041 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89cf0ce-a43e-40a5-b15f-2bcce1d4526d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:43.990323 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1189/2344] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:39:43 2022] POST /identity/v3/projects => generated 401 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:39:43.994789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08bf811-c03d-4d25-9090-788d4244825c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:44.005343 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08bf811-c03d-4d25-9090-788d4244825c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:44.065055 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cef1db-1a13-4ed4-891b-73d911d19884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:44.065819 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cef1db-1a13-4ed4-891b-73d911d19884 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:44.066035 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cef1db-1a13-4ed4-891b-73d911d19884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:44.066310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cef1db-1a13-4ed4-891b-73d911d19884 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:44.067031 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cef1db-1a13-4ed4-891b-73d911d19884 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:44.070088 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cef1db-1a13-4ed4-891b-73d911d19884 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:44.188322 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1156/2345] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:39:43 2022] POST /identity/v3/users => generated 301 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:39:44.189553 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89cf0ce-a43e-40a5-b15f-2bcce1d4526d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:44.200812 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89cf0ce-a43e-40a5-b15f-2bcce1d4526d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:44.206446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a6878-227b-478e-ba3f-ab8b0f366c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:44.207153 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a6878-227b-478e-ba3f-ab8b0f366c7f admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:44.207436 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a6878-227b-478e-ba3f-ab8b0f366c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:44.207749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a6878-227b-478e-ba3f-ab8b0f366c7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:44.210239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226a6878-227b-478e-ba3f-ab8b0f366c7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:44.210798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226a6878-227b-478e-ba3f-ab8b0f366c7f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:44.223913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1190/2346] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:39:44.229182 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cef1db-1a13-4ed4-891b-73d911d19884 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:44.239858 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cef1db-1a13-4ed4-891b-73d911d19884 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:44.245746 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee856bc-12bc-404c-91bd-8e3e0efe38a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:44.246469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee856bc-12bc-404c-91bd-8e3e0efe38a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:44.246765 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee856bc-12bc-404c-91bd-8e3e0efe38a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:44.247028 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee856bc-12bc-404c-91bd-8e3e0efe38a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:44.247869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee856bc-12bc-404c-91bd-8e3e0efe38a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:44.249503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee856bc-12bc-404c-91bd-8e3e0efe38a4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:44.262280 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1157/2347] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:39:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:39:44.266603 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a6878-227b-478e-ba3f-ab8b0f366c7f admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:44.277987 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a6878-227b-478e-ba3f-ab8b0f366c7f admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:44.285499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0f4c5-e94a-49cd-89d0-12f9c4d8f2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:44.286393 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0f4c5-e94a-49cd-89d0-12f9c4d8f2bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:44.286676 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0f4c5-e94a-49cd-89d0-12f9c4d8f2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:44.287018 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0f4c5-e94a-49cd-89d0-12f9c4d8f2bf admin admin] PATH_INFO: `/v3/projects/ac78a5bc18b946f3bd6f1164ceb5bef0/users/f440120678b84dd1a0cd564fe195437a/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:44.309236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0f4c5-e94a-49cd-89d0-12f9c4d8f2bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac78a5bc18b946f3bd6f1164ceb5bef0, user_id=f440120678b84dd1a0cd564fe195437a, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:44.312212 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0f4c5-e94a-49cd-89d0-12f9c4d8f2bf admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:44.389839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1191/2348] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:39:44 2022] PUT /identity/v3/projects/ac78a5bc18b946f3bd6f1164ceb5bef0/users/f440120678b84dd1a0cd564fe195437a/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:39:44.395216 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cdc134-b869-43b5-b5bc-2847c88f50b4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:44.395620 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cdc134-b869-43b5-b5bc-2847c88f50b4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:44.396047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cdc134-b869-43b5-b5bc-2847c88f50b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:44.416675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f8cdc134-b869-43b5-b5bc-2847c88f50b4 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:39:44.418231 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f8cdc134-b869-43b5-b5bc-2847c88f50b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:44.484908 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1158/2349] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:39:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:39:44.495628 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0f4c5-e94a-49cd-89d0-12f9c4d8f2bf admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:44.497378 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-baf0f4c5-e94a-49cd-89d0-12f9c4d8f2bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:44.508737 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:39:44.557900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0f4c5-e94a-49cd-89d0-12f9c4d8f2bf admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:44.562269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997f498-0c8f-47e3-a6b0-11d592146193 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:44.562924 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997f498-0c8f-47e3-a6b0-11d592146193 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:44.563210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997f498-0c8f-47e3-a6b0-11d592146193 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:44.563711 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997f498-0c8f-47e3-a6b0-11d592146193 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:44.574467 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7997f498-0c8f-47e3-a6b0-11d592146193 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:44.575994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7997f498-0c8f-47e3-a6b0-11d592146193 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:44.586808 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1192/2350] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:39:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:39:45.892225 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9d93e-02de-4ae2-b742-0989e19bcd8d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:45.892225 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9d93e-02de-4ae2-b742-0989e19bcd8d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:45.892225 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9d93e-02de-4ae2-b742-0989e19bcd8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:45.937820 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1dd9d93e-02de-4ae2-b742-0989e19bcd8d None None] MFA Rules not processed for user `f440120678b84dd1a0cd564fe195437a`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:39:45.939293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1dd9d93e-02de-4ae2-b742-0989e19bcd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:46.049783 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1159/2351] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:39:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:39:46.054698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997f498-0c8f-47e3-a6b0-11d592146193 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:46.073147 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997f498-0c8f-47e3-a6b0-11d592146193 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:46.082005 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d657fee-5546-4bdd-90fe-3624c8d9d152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:46.083916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d657fee-5546-4bdd-90fe-3624c8d9d152 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:46.084783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d657fee-5546-4bdd-90fe-3624c8d9d152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:46.085351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d657fee-5546-4bdd-90fe-3624c8d9d152 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:46.088218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d657fee-5546-4bdd-90fe-3624c8d9d152 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:46.092668 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d657fee-5546-4bdd-90fe-3624c8d9d152 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:46.175626 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1193/2352] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:39:46 2022] POST /identity/v3/projects => generated 403 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:39:46.179735 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9d93e-02de-4ae2-b742-0989e19bcd8d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:46.187602 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:39:46.196087 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9d93e-02de-4ae2-b742-0989e19bcd8d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:46.274952 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3b334-6a0e-4c81-8a11-dda535543a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:46.275982 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3b334-6a0e-4c81-8a11-dda535543a49 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:46.276792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3b334-6a0e-4c81-8a11-dda535543a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:46.277226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3b334-6a0e-4c81-8a11-dda535543a49 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:46.278107 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3b334-6a0e-4c81-8a11-dda535543a49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:46.280452 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3b334-6a0e-4c81-8a11-dda535543a49 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:46.392050 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1160/2353] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:39:46 2022] POST /identity/v3/users => generated 308 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:39:46.395783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d657fee-5546-4bdd-90fe-3624c8d9d152 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:46.407794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d657fee-5546-4bdd-90fe-3624c8d9d152 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:46.413340 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e59d9-048a-4337-8366-d2e3e7839290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:46.413995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e59d9-048a-4337-8366-d2e3e7839290 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:46.414250 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e59d9-048a-4337-8366-d2e3e7839290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:46.414527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4e59d9-048a-4337-8366-d2e3e7839290 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:46.415258 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e59d9-048a-4337-8366-d2e3e7839290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:46.417040 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4e59d9-048a-4337-8366-d2e3e7839290 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:46.430482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1194/2354] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:39:46.434421 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3b334-6a0e-4c81-8a11-dda535543a49 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:46.445840 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3b334-6a0e-4c81-8a11-dda535543a49 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:46.451823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d862953-a6a7-46c0-ba51-d2437dcbc7e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:46.452463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d862953-a6a7-46c0-ba51-d2437dcbc7e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:46.452758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d862953-a6a7-46c0-ba51-d2437dcbc7e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:46.453097 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d862953-a6a7-46c0-ba51-d2437dcbc7e0 admin admin] PATH_INFO: `/v3/projects/aaaa2e09b6e44ad6a8597bc0a3d3916c/users/f8bd561d26f1448398b45bdbafd90189/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:46.471932 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d862953-a6a7-46c0-ba51-d2437dcbc7e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaaa2e09b6e44ad6a8597bc0a3d3916c, user_id=f8bd561d26f1448398b45bdbafd90189, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:46.474690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d862953-a6a7-46c0-ba51-d2437dcbc7e0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:46.567748 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1161/2355] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:39:46 2022] PUT /identity/v3/projects/aaaa2e09b6e44ad6a8597bc0a3d3916c/users/f8bd561d26f1448398b45bdbafd90189/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:39:46.571621 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e59d9-048a-4337-8366-d2e3e7839290 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:46.581969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4e59d9-048a-4337-8366-d2e3e7839290 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:46.651431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d081b-eeab-4f70-9f19-932b2ce0ab80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:46.652170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d081b-eeab-4f70-9f19-932b2ce0ab80 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:46.652458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d081b-eeab-4f70-9f19-932b2ce0ab80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:46.652737 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d081b-eeab-4f70-9f19-932b2ce0ab80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:46.653470 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d081b-eeab-4f70-9f19-932b2ce0ab80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:46.655426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d081b-eeab-4f70-9f19-932b2ce0ab80 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:46.667963 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1195/2356] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:39:46 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:39:46.672016 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d862953-a6a7-46c0-ba51-d2437dcbc7e0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:46.683755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d862953-a6a7-46c0-ba51-d2437dcbc7e0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:46.689401 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6f74b-260b-421f-a968-4d86fcc846e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:46.690321 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6f74b-260b-421f-a968-4d86fcc846e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:46.690585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6f74b-260b-421f-a968-4d86fcc846e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:46.695297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6f74b-260b-421f-a968-4d86fcc846e9 admin admin] PATH_INFO: `/v3/projects/aaaa2e09b6e44ad6a8597bc0a3d3916c/users/f8bd561d26f1448398b45bdbafd90189/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:46.695297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6f74b-260b-421f-a968-4d86fcc846e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaaa2e09b6e44ad6a8597bc0a3d3916c, user_id=f8bd561d26f1448398b45bdbafd90189, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:46.695747 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6f74b-260b-421f-a968-4d86fcc846e9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:46.707892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1162/2357] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:39:46 2022] PUT /identity/v3/projects/aaaa2e09b6e44ad6a8597bc0a3d3916c/users/f8bd561d26f1448398b45bdbafd90189/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:39:48.437645 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72472c0c-4d99-4756-83ad-567ff43a6a62 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:48.438231 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72472c0c-4d99-4756-83ad-567ff43a6a62 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:48.438445 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72472c0c-4d99-4756-83ad-567ff43a6a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:48.496169 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-72472c0c-4d99-4756-83ad-567ff43a6a62 None None] MFA Rules not processed for user `f8bd561d26f1448398b45bdbafd90189`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:39:48.497565 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-72472c0c-4d99-4756-83ad-567ff43a6a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:48.599434 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:39:48.600960 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1196/2358] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:39:48 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:39:49.989912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6f74b-260b-421f-a968-4d86fcc846e9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:50.056301 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6f74b-260b-421f-a968-4d86fcc846e9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:50.061463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b632d07-9602-42d8-b77f-1946122308c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:50.062355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b632d07-9602-42d8-b77f-1946122308c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:50.062642 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b632d07-9602-42d8-b77f-1946122308c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:50.062916 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b632d07-9602-42d8-b77f-1946122308c5 admin admin] PATH_INFO: `/v3/users/f440120678b84dd1a0cd564fe195437a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:50.064256 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b632d07-9602-42d8-b77f-1946122308c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f440120678b84dd1a0cd564fe195437a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:50.066183 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b632d07-9602-42d8-b77f-1946122308c5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:50.284826 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2b632d07-9602-42d8-b77f-1946122308c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f440120678b84dd1a0cd564fe195437a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:50.285407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2b632d07-9602-42d8-b77f-1946122308c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f440120678b84dd1a0cd564fe195437a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:50.343281 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2b632d07-9602-42d8-b77f-1946122308c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f440120678b84dd1a0cd564fe195437a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:50.345972 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:39:50.352703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2b632d07-9602-42d8-b77f-1946122308c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f440120678b84dd1a0cd564fe195437a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:50.353527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2b632d07-9602-42d8-b77f-1946122308c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f440120678b84dd1a0cd564fe195437a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:50.371958 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1163/2359] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:39:49 2022] DELETE /identity/v3/users/f440120678b84dd1a0cd564fe195437a => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:39:50.648842 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72472c0c-4d99-4756-83ad-567ff43a6a62 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:50.659734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72472c0c-4d99-4756-83ad-567ff43a6a62 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:50.726520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602577e6-39ff-455c-9245-99a441627cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:50.727918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602577e6-39ff-455c-9245-99a441627cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:50.728436 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602577e6-39ff-455c-9245-99a441627cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:50.728842 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602577e6-39ff-455c-9245-99a441627cde admin admin] PATH_INFO: `/v3/projects/ac78a5bc18b946f3bd6f1164ceb5bef0` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:50.730305 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602577e6-39ff-455c-9245-99a441627cde admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac78a5bc18b946f3bd6f1164ceb5bef0)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:50.732574 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602577e6-39ff-455c-9245-99a441627cde admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:50.842292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-602577e6-39ff-455c-9245-99a441627cde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac78a5bc18b946f3bd6f1164ceb5bef0', 'request_id': 'req-602577e6-39ff-455c-9245-99a441627cde'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:50.920334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-602577e6-39ff-455c-9245-99a441627cde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac78a5bc18b946f3bd6f1164ceb5bef0', 'request_id': 'req-602577e6-39ff-455c-9245-99a441627cde'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:50.927900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-602577e6-39ff-455c-9245-99a441627cde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac78a5bc18b946f3bd6f1164ceb5bef0', 'request_id': 'req-602577e6-39ff-455c-9245-99a441627cde'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:50.941598 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-602577e6-39ff-455c-9245-99a441627cde admin admin] The token cache is being invalidate because project ac78a5bc18b946f3bd6f1164ceb5bef0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:39:50.941890 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-602577e6-39ff-455c-9245-99a441627cde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:50.942556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-602577e6-39ff-455c-9245-99a441627cde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:50.944946 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1197/2360] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:39:50 2022] DELETE /identity/v3/projects/ac78a5bc18b946f3bd6f1164ceb5bef0 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:39:50.948844 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b632d07-9602-42d8-b77f-1946122308c5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:50.950347 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2b632d07-9602-42d8-b77f-1946122308c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:50.965412 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b632d07-9602-42d8-b77f-1946122308c5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:51.028323 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb41948-7f01-4124-9ba0-1a9977bfafc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:51.029202 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb41948-7f01-4124-9ba0-1a9977bfafc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:51.029482 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb41948-7f01-4124-9ba0-1a9977bfafc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:51.029753 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb41948-7f01-4124-9ba0-1a9977bfafc4 admin admin] PATH_INFO: `/v3/users/f8bd561d26f1448398b45bdbafd90189` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:51.030932 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb41948-7f01-4124-9ba0-1a9977bfafc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8bd561d26f1448398b45bdbafd90189)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:51.032857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb41948-7f01-4124-9ba0-1a9977bfafc4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:51.261636 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ceb41948-7f01-4124-9ba0-1a9977bfafc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8bd561d26f1448398b45bdbafd90189'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:51.262249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ceb41948-7f01-4124-9ba0-1a9977bfafc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8bd561d26f1448398b45bdbafd90189'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:51.348903 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ceb41948-7f01-4124-9ba0-1a9977bfafc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8bd561d26f1448398b45bdbafd90189'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:51.357447 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ceb41948-7f01-4124-9ba0-1a9977bfafc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8bd561d26f1448398b45bdbafd90189'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:51.358118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ceb41948-7f01-4124-9ba0-1a9977bfafc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8bd561d26f1448398b45bdbafd90189'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:51.376432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1164/2361] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:39:50 2022] DELETE /identity/v3/users/f8bd561d26f1448398b45bdbafd90189 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:39:51.619324 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602577e6-39ff-455c-9245-99a441627cde admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:51.620792 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-602577e6-39ff-455c-9245-99a441627cde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:51.646287 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602577e6-39ff-455c-9245-99a441627cde admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:51.694629 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326bee07-8f6d-46da-b14c-6e039a8da76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:51.696037 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bee07-8f6d-46da-b14c-6e039a8da76b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:51.696318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bee07-8f6d-46da-b14c-6e039a8da76b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:51.696601 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bee07-8f6d-46da-b14c-6e039a8da76b admin admin] PATH_INFO: `/v3/projects/aaaa2e09b6e44ad6a8597bc0a3d3916c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:51.697775 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326bee07-8f6d-46da-b14c-6e039a8da76b admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaaa2e09b6e44ad6a8597bc0a3d3916c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:51.699736 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326bee07-8f6d-46da-b14c-6e039a8da76b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:51.812211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:39:51.825748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-326bee07-8f6d-46da-b14c-6e039a8da76b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaaa2e09b6e44ad6a8597bc0a3d3916c', 'request_id': 'req-326bee07-8f6d-46da-b14c-6e039a8da76b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:51.995035 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-326bee07-8f6d-46da-b14c-6e039a8da76b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaaa2e09b6e44ad6a8597bc0a3d3916c', 'request_id': 'req-326bee07-8f6d-46da-b14c-6e039a8da76b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:52.001909 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-326bee07-8f6d-46da-b14c-6e039a8da76b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaaa2e09b6e44ad6a8597bc0a3d3916c', 'request_id': 'req-326bee07-8f6d-46da-b14c-6e039a8da76b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:52.014250 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-326bee07-8f6d-46da-b14c-6e039a8da76b admin admin] The token cache is being invalidate because project aaaa2e09b6e44ad6a8597bc0a3d3916c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:39:52.014547 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-326bee07-8f6d-46da-b14c-6e039a8da76b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:52.015222 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-326bee07-8f6d-46da-b14c-6e039a8da76b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:52.016960 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1198/2362] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:39:51 2022] DELETE /identity/v3/projects/aaaa2e09b6e44ad6a8597bc0a3d3916c => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:39:52.021949 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052f9ae-e9b0-4bea-9c7d-98cb55e04a68 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:52.022249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052f9ae-e9b0-4bea-9c7d-98cb55e04a68 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:52.027396 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052f9ae-e9b0-4bea-9c7d-98cb55e04a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:52.040637 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5052f9ae-e9b0-4bea-9c7d-98cb55e04a68 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:39:52.041945 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5052f9ae-e9b0-4bea-9c7d-98cb55e04a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:52.093819 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:39:52.108036 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1165/2363] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:39:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:39:52.117071 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc8148-111f-4573-bba9-a404fa751c1d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:52.118073 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc8148-111f-4573-bba9-a404fa751c1d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:52.118537 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc8148-111f-4573-bba9-a404fa751c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:52.141860 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-09cc8148-111f-4573-bba9-a404fa751c1d None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:39:52.143072 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-09cc8148-111f-4573-bba9-a404fa751c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:52.151128 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1199/2364] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:39:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:39:52.155557 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052f9ae-e9b0-4bea-9c7d-98cb55e04a68 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:52.165651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052f9ae-e9b0-4bea-9c7d-98cb55e04a68 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:52.170296 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2061e7c-11e7-4a03-9fd2-843cd3d2a90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:52.170968 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2061e7c-11e7-4a03-9fd2-843cd3d2a90a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:52.171249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2061e7c-11e7-4a03-9fd2-843cd3d2a90a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:52.171558 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2061e7c-11e7-4a03-9fd2-843cd3d2a90a admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:52.172285 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2061e7c-11e7-4a03-9fd2-843cd3d2a90a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:39:52.172590 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2061e7c-11e7-4a03-9fd2-843cd3d2a90a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:52.174189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2061e7c-11e7-4a03-9fd2-843cd3d2a90a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:52.187176 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1166/2365] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:39:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:39:52.191551 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc8148-111f-4573-bba9-a404fa751c1d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:52.201396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc8148-111f-4573-bba9-a404fa751c1d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:52.206117 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f6acb-d228-412f-afc6-ab6c40aa19f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:52.206777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f6acb-d228-412f-afc6-ab6c40aa19f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:52.207062 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f6acb-d228-412f-afc6-ab6c40aa19f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:52.207342 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f6acb-d228-412f-afc6-ab6c40aa19f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:52.208104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f6acb-d228-412f-afc6-ab6c40aa19f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:52.210043 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f6acb-d228-412f-afc6-ab6c40aa19f6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:52.322844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1200/2366] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:39:52 2022] POST /identity/v3/projects => generated 371 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:39:52.326927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2061e7c-11e7-4a03-9fd2-843cd3d2a90a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:52.338187 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2061e7c-11e7-4a03-9fd2-843cd3d2a90a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:52.398125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00329797-84ac-4be7-8d56-327d801a4f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:52.399000 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00329797-84ac-4be7-8d56-327d801a4f93 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:52.399313 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00329797-84ac-4be7-8d56-327d801a4f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:52.399612 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00329797-84ac-4be7-8d56-327d801a4f93 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:52.400338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00329797-84ac-4be7-8d56-327d801a4f93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:52.402283 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00329797-84ac-4be7-8d56-327d801a4f93 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:52.503460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1167/2367] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:39:52 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:39:52.508229 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f6acb-d228-412f-afc6-ab6c40aa19f6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:52.519530 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f6acb-d228-412f-afc6-ab6c40aa19f6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:52.525185 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e41221-c789-46ac-83ec-43a210c7bef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:52.525852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e41221-c789-46ac-83ec-43a210c7bef2 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:52.526141 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e41221-c789-46ac-83ec-43a210c7bef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:52.526417 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e41221-c789-46ac-83ec-43a210c7bef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:52.527193 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e41221-c789-46ac-83ec-43a210c7bef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:52.528921 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e41221-c789-46ac-83ec-43a210c7bef2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:52.542246 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1201/2368] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:39:52.546531 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00329797-84ac-4be7-8d56-327d801a4f93 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:52.556724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00329797-84ac-4be7-8d56-327d801a4f93 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:52.561517 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42adc3e4-df36-464f-b2fd-937236f78c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:52.562176 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adc3e4-df36-464f-b2fd-937236f78c2a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:52.562458 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adc3e4-df36-464f-b2fd-937236f78c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:52.562735 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adc3e4-df36-464f-b2fd-937236f78c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:52.563449 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42adc3e4-df36-464f-b2fd-937236f78c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:52.565063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42adc3e4-df36-464f-b2fd-937236f78c2a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:52.575538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1168/2369] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:39:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:39:52.579963 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e41221-c789-46ac-83ec-43a210c7bef2 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:52.590800 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e41221-c789-46ac-83ec-43a210c7bef2 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:52.595824 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28359b7-9eef-4290-b70b-95558e26fe2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:52.596724 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28359b7-9eef-4290-b70b-95558e26fe2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:52.597040 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28359b7-9eef-4290-b70b-95558e26fe2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:52.597313 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28359b7-9eef-4290-b70b-95558e26fe2a admin admin] PATH_INFO: `/v3/projects/76974991c22b4e6aa7eaadf92ea823f3/users/e472aed7b21f4cbebede88c13421626b/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:52.613552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28359b7-9eef-4290-b70b-95558e26fe2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=76974991c22b4e6aa7eaadf92ea823f3, user_id=e472aed7b21f4cbebede88c13421626b, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:52.616240 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28359b7-9eef-4290-b70b-95558e26fe2a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:52.738082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1202/2370] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:39:52 2022] PUT /identity/v3/projects/76974991c22b4e6aa7eaadf92ea823f3/users/e472aed7b21f4cbebede88c13421626b/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:39:52.740665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a4eba-102a-4195-a0f4-a927f9add923 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:52.741069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a4eba-102a-4195-a0f4-a927f9add923 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:52.742455 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a4eba-102a-4195-a0f4-a927f9add923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:52.771942 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-522a4eba-102a-4195-a0f4-a927f9add923 None None] MFA Rules not processed for user `e472aed7b21f4cbebede88c13421626b`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:39:52.773284 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-522a4eba-102a-4195-a0f4-a927f9add923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:52.845525 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1169/2371] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:39:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:39:52.850192 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28359b7-9eef-4290-b70b-95558e26fe2a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:52.917928 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28359b7-9eef-4290-b70b-95558e26fe2a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:52.923001 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b2995-c271-45d7-b43a-b3bd9fa9a067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:52.923728 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b2995-c271-45d7-b43a-b3bd9fa9a067 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:52.923985 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b2995-c271-45d7-b43a-b3bd9fa9a067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:52.924269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b2995-c271-45d7-b43a-b3bd9fa9a067 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:52.924979 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b2995-c271-45d7-b43a-b3bd9fa9a067 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:52.926919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b2995-c271-45d7-b43a-b3bd9fa9a067 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:53.036798 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1203/2372] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:39:52 2022] POST /identity/v3/projects => generated 367 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:39:53.040951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a4eba-102a-4195-a0f4-a927f9add923 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:53.052991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a4eba-102a-4195-a0f4-a927f9add923 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:53.103974 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:39:53.115710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9840e-8ad0-4cd8-8bcd-6a2a0ed0a329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:53.116388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9840e-8ad0-4cd8-8bcd-6a2a0ed0a329 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:53.121005 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9840e-8ad0-4cd8-8bcd-6a2a0ed0a329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:53.121005 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9840e-8ad0-4cd8-8bcd-6a2a0ed0a329 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:53.121005 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b9840e-8ad0-4cd8-8bcd-6a2a0ed0a329 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:53.121005 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b9840e-8ad0-4cd8-8bcd-6a2a0ed0a329 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:53.189952 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1170/2373] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:39:53 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:39:53.193989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b2995-c271-45d7-b43a-b3bd9fa9a067 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:53.205036 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b2995-c271-45d7-b43a-b3bd9fa9a067 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:53.209791 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30fab7-2e7e-4f76-ac6f-33bcaa00e75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:53.210459 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30fab7-2e7e-4f76-ac6f-33bcaa00e75c admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:53.210747 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30fab7-2e7e-4f76-ac6f-33bcaa00e75c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:53.211420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30fab7-2e7e-4f76-ac6f-33bcaa00e75c admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:53.212261 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b30fab7-2e7e-4f76-ac6f-33bcaa00e75c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:53.214464 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b30fab7-2e7e-4f76-ac6f-33bcaa00e75c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:53.229585 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1204/2374] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:39:53.230553 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9840e-8ad0-4cd8-8bcd-6a2a0ed0a329 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:53.241038 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b9840e-8ad0-4cd8-8bcd-6a2a0ed0a329 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:53.246841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373dadf0-cb15-4409-8421-ac09a7f036fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:53.247783 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dadf0-cb15-4409-8421-ac09a7f036fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:53.248062 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dadf0-cb15-4409-8421-ac09a7f036fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:53.248338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dadf0-cb15-4409-8421-ac09a7f036fd admin admin] PATH_INFO: `/v3/projects/a2ebd2bad132493eaa47d136d002c336/users/462936caaa2d45abbeed0d3e0b869013/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:53.266518 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373dadf0-cb15-4409-8421-ac09a7f036fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2ebd2bad132493eaa47d136d002c336, user_id=462936caaa2d45abbeed0d3e0b869013, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:53.269757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373dadf0-cb15-4409-8421-ac09a7f036fd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:53.375615 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1171/2375] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:39:53 2022] PUT /identity/v3/projects/a2ebd2bad132493eaa47d136d002c336/users/462936caaa2d45abbeed0d3e0b869013/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:39:53.378180 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30fab7-2e7e-4f76-ac6f-33bcaa00e75c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:53.392773 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30fab7-2e7e-4f76-ac6f-33bcaa00e75c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:53.420375 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:39:53.452725 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35367d00-fd0d-403c-95a1-1c82a4803cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:53.453387 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35367d00-fd0d-403c-95a1-1c82a4803cad admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:53.453673 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35367d00-fd0d-403c-95a1-1c82a4803cad admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:53.453951 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35367d00-fd0d-403c-95a1-1c82a4803cad admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:53.454689 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35367d00-fd0d-403c-95a1-1c82a4803cad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:53.456402 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35367d00-fd0d-403c-95a1-1c82a4803cad admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:53.467107 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1205/2376] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:39:53 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:39:53.471370 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373dadf0-cb15-4409-8421-ac09a7f036fd admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:53.482494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373dadf0-cb15-4409-8421-ac09a7f036fd admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:53.487156 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82c98a-d559-482b-86ba-d490afb7e868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:53.488071 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82c98a-d559-482b-86ba-d490afb7e868 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:53.488354 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82c98a-d559-482b-86ba-d490afb7e868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:53.488629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82c98a-d559-482b-86ba-d490afb7e868 admin admin] PATH_INFO: `/v3/projects/a2ebd2bad132493eaa47d136d002c336/users/462936caaa2d45abbeed0d3e0b869013/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:53.490642 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82c98a-d559-482b-86ba-d490afb7e868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2ebd2bad132493eaa47d136d002c336, user_id=462936caaa2d45abbeed0d3e0b869013, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:53.493918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82c98a-d559-482b-86ba-d490afb7e868 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:53.504495 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1172/2377] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:39:53 2022] PUT /identity/v3/projects/a2ebd2bad132493eaa47d136d002c336/users/462936caaa2d45abbeed0d3e0b869013/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:39:53.516669 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ac04c-e374-48f0-89e3-1d402fc55dfa None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:53.517050 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ac04c-e374-48f0-89e3-1d402fc55dfa None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:53.517339 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ac04c-e374-48f0-89e3-1d402fc55dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:53.548705 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c54ac04c-e374-48f0-89e3-1d402fc55dfa None None] MFA Rules not processed for user `462936caaa2d45abbeed0d3e0b869013`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:39:53.549906 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c54ac04c-e374-48f0-89e3-1d402fc55dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:53.622094 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1206/2378] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:39:53 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:39:53.634392 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82c98a-d559-482b-86ba-d490afb7e868 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:53.636104 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9d82c98a-d559-482b-86ba-d490afb7e868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:53.696462 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82c98a-d559-482b-86ba-d490afb7e868 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:53.700116 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf36f8-b078-4329-a8d4-4b03aef61c0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:53.700752 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf36f8-b078-4329-a8d4-4b03aef61c0a service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:53.701031 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf36f8-b078-4329-a8d4-4b03aef61c0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:53.701303 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf36f8-b078-4329-a8d4-4b03aef61c0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:53.710422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf36f8-b078-4329-a8d4-4b03aef61c0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:53.712020 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf36f8-b078-4329-a8d4-4b03aef61c0a service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:53.721381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1173/2379] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:39:53 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:39:54.835267 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ac04c-e374-48f0-89e3-1d402fc55dfa None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:54.914050 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ac04c-e374-48f0-89e3-1d402fc55dfa None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:54.919193 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b8b02-0490-4823-b967-89d654a599b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:54.920342 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b8b02-0490-4823-b967-89d654a599b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:54.920694 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b8b02-0490-4823-b967-89d654a599b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:54.920983 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b8b02-0490-4823-b967-89d654a599b6 admin admin] PATH_INFO: `/v3/users/e472aed7b21f4cbebede88c13421626b` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:54.922261 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b8b02-0490-4823-b967-89d654a599b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e472aed7b21f4cbebede88c13421626b)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:54.925450 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052b8b02-0490-4823-b967-89d654a599b6 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:55.186309 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-052b8b02-0490-4823-b967-89d654a599b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e472aed7b21f4cbebede88c13421626b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:55.187223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-052b8b02-0490-4823-b967-89d654a599b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e472aed7b21f4cbebede88c13421626b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:55.192412 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:39:55.196095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-052b8b02-0490-4823-b967-89d654a599b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e472aed7b21f4cbebede88c13421626b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:55.372327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-052b8b02-0490-4823-b967-89d654a599b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e472aed7b21f4cbebede88c13421626b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:55.373059 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-052b8b02-0490-4823-b967-89d654a599b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e472aed7b21f4cbebede88c13421626b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:55.398833 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1207/2380] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:39:54 2022] DELETE /identity/v3/users/e472aed7b21f4cbebede88c13421626b => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:39:55.403323 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2bd94-7457-43be-87bb-68082e3501be None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:55.403886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2bd94-7457-43be-87bb-68082e3501be None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:55.403991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e2bd94-7457-43be-87bb-68082e3501be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:55.422615 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-71e2bd94-7457-43be-87bb-68082e3501be None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:39:55.423854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-71e2bd94-7457-43be-87bb-68082e3501be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:55.496049 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1174/2381] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:39:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:39:55.508483 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b8b02-0490-4823-b967-89d654a599b6 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:55.510528 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-052b8b02-0490-4823-b967-89d654a599b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:55.573916 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052b8b02-0490-4823-b967-89d654a599b6 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:55.577873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6055593-cbb8-435e-b511-835424d0d286 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:55.578550 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6055593-cbb8-435e-b511-835424d0d286 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:55.578835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6055593-cbb8-435e-b511-835424d0d286 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:55.579115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6055593-cbb8-435e-b511-835424d0d286 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:55.589946 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6055593-cbb8-435e-b511-835424d0d286 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:55.591669 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6055593-cbb8-435e-b511-835424d0d286 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:55.604587 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1208/2382] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:39:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:39:56.089469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2bd94-7457-43be-87bb-68082e3501be None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:56.093758 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-71e2bd94-7457-43be-87bb-68082e3501be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:56.106303 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e2bd94-7457-43be-87bb-68082e3501be None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:56.112288 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e83f84-1afc-42d7-94b2-a98c3c1ac4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:56.113556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e83f84-1afc-42d7-94b2-a98c3c1ac4cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:56.114874 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e83f84-1afc-42d7-94b2-a98c3c1ac4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:56.115367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e83f84-1afc-42d7-94b2-a98c3c1ac4cf admin admin] PATH_INFO: `/v3/projects/76974991c22b4e6aa7eaadf92ea823f3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:56.116682 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e83f84-1afc-42d7-94b2-a98c3c1ac4cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=76974991c22b4e6aa7eaadf92ea823f3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:56.119572 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e83f84-1afc-42d7-94b2-a98c3c1ac4cf admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:56.224863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:39:56.233415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-98e83f84-1afc-42d7-94b2-a98c3c1ac4cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76974991c22b4e6aa7eaadf92ea823f3', 'request_id': 'req-98e83f84-1afc-42d7-94b2-a98c3c1ac4cf'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:56.243708 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-98e83f84-1afc-42d7-94b2-a98c3c1ac4cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76974991c22b4e6aa7eaadf92ea823f3', 'request_id': 'req-98e83f84-1afc-42d7-94b2-a98c3c1ac4cf'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:56.252379 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-98e83f84-1afc-42d7-94b2-a98c3c1ac4cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76974991c22b4e6aa7eaadf92ea823f3', 'request_id': 'req-98e83f84-1afc-42d7-94b2-a98c3c1ac4cf'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:56.328369 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-98e83f84-1afc-42d7-94b2-a98c3c1ac4cf admin admin] The token cache is being invalidate because project 76974991c22b4e6aa7eaadf92ea823f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:39:56.328627 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-98e83f84-1afc-42d7-94b2-a98c3c1ac4cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:56.329297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-98e83f84-1afc-42d7-94b2-a98c3c1ac4cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:56.330840 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1175/2383] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:39:56 2022] DELETE /identity/v3/projects/76974991c22b4e6aa7eaadf92ea823f3 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:39:56.335769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6055593-cbb8-435e-b511-835424d0d286 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:56.346026 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6055593-cbb8-435e-b511-835424d0d286 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:56.423150 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ee64e6-ccc0-429e-927c-a292556dc040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:56.426480 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee64e6-ccc0-429e-927c-a292556dc040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:56.426480 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee64e6-ccc0-429e-927c-a292556dc040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:56.426480 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ee64e6-ccc0-429e-927c-a292556dc040 admin admin] PATH_INFO: `/v3/users/462936caaa2d45abbeed0d3e0b869013` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:56.426480 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee64e6-ccc0-429e-927c-a292556dc040 admin admin] RBAC: Authorizing `identity:delete_user(user_id=462936caaa2d45abbeed0d3e0b869013)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:56.427788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ee64e6-ccc0-429e-927c-a292556dc040 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:56.677763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c5ee64e6-ccc0-429e-927c-a292556dc040 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '462936caaa2d45abbeed0d3e0b869013'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:56.678508 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c5ee64e6-ccc0-429e-927c-a292556dc040 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '462936caaa2d45abbeed0d3e0b869013'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:56.687027 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c5ee64e6-ccc0-429e-927c-a292556dc040 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '462936caaa2d45abbeed0d3e0b869013'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:56.778138 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c5ee64e6-ccc0-429e-927c-a292556dc040 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '462936caaa2d45abbeed0d3e0b869013'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:56.778788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-c5ee64e6-ccc0-429e-927c-a292556dc040 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '462936caaa2d45abbeed0d3e0b869013'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:56.787698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:39:56.798711 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1209/2384] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:39:56 2022] DELETE /identity/v3/users/462936caaa2d45abbeed0d3e0b869013 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:39:57.363624 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e83f84-1afc-42d7-94b2-a98c3c1ac4cf admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:57.365349 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-98e83f84-1afc-42d7-94b2-a98c3c1ac4cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:57.393019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e83f84-1afc-42d7-94b2-a98c3c1ac4cf admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:57.479323 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b80d05-c449-42ea-b16c-21d78cdd31ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:57.480593 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b80d05-c449-42ea-b16c-21d78cdd31ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:57.480977 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b80d05-c449-42ea-b16c-21d78cdd31ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:57.481747 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b80d05-c449-42ea-b16c-21d78cdd31ea admin admin] PATH_INFO: `/v3/projects/a2ebd2bad132493eaa47d136d002c336` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:57.487768 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b80d05-c449-42ea-b16c-21d78cdd31ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2ebd2bad132493eaa47d136d002c336)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:57.490468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b80d05-c449-42ea-b16c-21d78cdd31ea admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:57.639572 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-71b80d05-c449-42ea-b16c-21d78cdd31ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2ebd2bad132493eaa47d136d002c336', 'request_id': 'req-71b80d05-c449-42ea-b16c-21d78cdd31ea'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:57.646389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-71b80d05-c449-42ea-b16c-21d78cdd31ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2ebd2bad132493eaa47d136d002c336', 'request_id': 'req-71b80d05-c449-42ea-b16c-21d78cdd31ea'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:57.654756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-71b80d05-c449-42ea-b16c-21d78cdd31ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2ebd2bad132493eaa47d136d002c336', 'request_id': 'req-71b80d05-c449-42ea-b16c-21d78cdd31ea'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:57.755965 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-71b80d05-c449-42ea-b16c-21d78cdd31ea admin admin] The token cache is being invalidate because project a2ebd2bad132493eaa47d136d002c336 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:39:57.756260 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-71b80d05-c449-42ea-b16c-21d78cdd31ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:57.756982 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-71b80d05-c449-42ea-b16c-21d78cdd31ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:57.759432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1176/2385] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:39:57 2022] DELETE /identity/v3/projects/a2ebd2bad132493eaa47d136d002c336 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:39:57.765249 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d6f14-6707-451b-b08e-67b432b1441e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:57.765804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d6f14-6707-451b-b08e-67b432b1441e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:57.765904 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4d6f14-6707-451b-b08e-67b432b1441e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:57.791728 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ca4d6f14-6707-451b-b08e-67b432b1441e None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:39:57.793069 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ca4d6f14-6707-451b-b08e-67b432b1441e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:57.876547 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1210/2386] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:39:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:39:57.886600 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ada30-99ae-474b-afba-a3c70c4681a8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:57.886989 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ada30-99ae-474b-afba-a3c70c4681a8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:57.887343 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ada30-99ae-474b-afba-a3c70c4681a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:57.908598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-922ada30-99ae-474b-afba-a3c70c4681a8 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:39:57.909943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-922ada30-99ae-474b-afba-a3c70c4681a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:57.919590 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1177/2387] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:39:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:39:57.924472 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d6f14-6707-451b-b08e-67b432b1441e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:57.936911 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4d6f14-6707-451b-b08e-67b432b1441e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:57.942835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ecffd6-eeec-4c94-af29-e2628822141e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:57.943783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ecffd6-eeec-4c94-af29-e2628822141e admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:57.944188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ecffd6-eeec-4c94-af29-e2628822141e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:57.944470 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ecffd6-eeec-4c94-af29-e2628822141e admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:57.945219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ecffd6-eeec-4c94-af29-e2628822141e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:39:57.945656 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ecffd6-eeec-4c94-af29-e2628822141e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:57.947902 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ecffd6-eeec-4c94-af29-e2628822141e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:57.961444 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1211/2388] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:39:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:39:57.966127 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ada30-99ae-474b-afba-a3c70c4681a8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:57.976011 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ada30-99ae-474b-afba-a3c70c4681a8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:57.982907 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504a653-032c-4dd3-bf3b-0540903b2113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:57.982907 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504a653-032c-4dd3-bf3b-0540903b2113 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:57.983269 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504a653-032c-4dd3-bf3b-0540903b2113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:57.983269 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504a653-032c-4dd3-bf3b-0540903b2113 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:57.983981 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f504a653-032c-4dd3-bf3b-0540903b2113 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:57.985948 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f504a653-032c-4dd3-bf3b-0540903b2113 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:58.116205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1178/2389] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:39:57 2022] POST /identity/v3/projects => generated 387 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:39:58.118498 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ecffd6-eeec-4c94-af29-e2628822141e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:58.131019 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ecffd6-eeec-4c94-af29-e2628822141e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:58.200796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471708e3-859e-4f2b-9a64-37adcd6bfd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:58.201597 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471708e3-859e-4f2b-9a64-37adcd6bfd0e admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:58.201958 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471708e3-859e-4f2b-9a64-37adcd6bfd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:58.202310 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471708e3-859e-4f2b-9a64-37adcd6bfd0e admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:58.203080 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471708e3-859e-4f2b-9a64-37adcd6bfd0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:58.205092 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471708e3-859e-4f2b-9a64-37adcd6bfd0e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:58.294437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1212/2390] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:39:58 2022] POST /identity/v3/users => generated 294 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:39:58.298694 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504a653-032c-4dd3-bf3b-0540903b2113 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:58.310577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f504a653-032c-4dd3-bf3b-0540903b2113 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:58.316041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43454f-e311-45a6-a352-1bedf9f759d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:58.316654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43454f-e311-45a6-a352-1bedf9f759d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:58.316943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43454f-e311-45a6-a352-1bedf9f759d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:58.317225 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43454f-e311-45a6-a352-1bedf9f759d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:58.317962 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43454f-e311-45a6-a352-1bedf9f759d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:58.319660 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43454f-e311-45a6-a352-1bedf9f759d9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:58.332889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1179/2391] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:39:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:39:58.337423 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471708e3-859e-4f2b-9a64-37adcd6bfd0e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:58.350651 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471708e3-859e-4f2b-9a64-37adcd6bfd0e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:58.357135 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4811779c-4bf6-404e-b59c-2ee880d6cc8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:58.361925 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811779c-4bf6-404e-b59c-2ee880d6cc8a admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:58.361925 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811779c-4bf6-404e-b59c-2ee880d6cc8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:58.361925 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4811779c-4bf6-404e-b59c-2ee880d6cc8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:58.361925 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811779c-4bf6-404e-b59c-2ee880d6cc8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:58.361925 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4811779c-4bf6-404e-b59c-2ee880d6cc8a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:58.373550 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1213/2392] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:39:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:39:58.378306 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43454f-e311-45a6-a352-1bedf9f759d9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:58.390521 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43454f-e311-45a6-a352-1bedf9f759d9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:58.395690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18f38a-a7c0-4a16-98b0-d7fe3ccbb5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:58.396067 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18f38a-a7c0-4a16-98b0-d7fe3ccbb5a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:58.396351 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18f38a-a7c0-4a16-98b0-d7fe3ccbb5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:58.396622 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18f38a-a7c0-4a16-98b0-d7fe3ccbb5a3 admin admin] PATH_INFO: `/v3/projects/0e1b3a8273d949fab03bdd973894a9d6/users/a3fb1dfbde384d6987068bda8596fd95/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:58.402471 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:39:58.415491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18f38a-a7c0-4a16-98b0-d7fe3ccbb5a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e1b3a8273d949fab03bdd973894a9d6, user_id=a3fb1dfbde384d6987068bda8596fd95, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:58.418239 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18f38a-a7c0-4a16-98b0-d7fe3ccbb5a3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:58.517059 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1180/2393] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:39:58 2022] PUT /identity/v3/projects/0e1b3a8273d949fab03bdd973894a9d6/users/a3fb1dfbde384d6987068bda8596fd95/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:39:58.528779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee06c89-4651-41d5-8d1d-95d2632c41fb None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:58.529137 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee06c89-4651-41d5-8d1d-95d2632c41fb None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:58.529360 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee06c89-4651-41d5-8d1d-95d2632c41fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:58.561047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-4ee06c89-4651-41d5-8d1d-95d2632c41fb None None] MFA Rules not processed for user `a3fb1dfbde384d6987068bda8596fd95`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:39:58.562333 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4ee06c89-4651-41d5-8d1d-95d2632c41fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:58.612428 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:39:58.632827 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1214/2394] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:39:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:39:58.637560 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18f38a-a7c0-4a16-98b0-d7fe3ccbb5a3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:58.706807 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18f38a-a7c0-4a16-98b0-d7fe3ccbb5a3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:58.712906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8d383-064c-4bea-a037-3502e193d7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:58.713458 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8d383-064c-4bea-a037-3502e193d7ed admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:58.713835 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8d383-064c-4bea-a037-3502e193d7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:58.714071 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8d383-064c-4bea-a037-3502e193d7ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:58.714775 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8d383-064c-4bea-a037-3502e193d7ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:58.716748 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8d383-064c-4bea-a037-3502e193d7ed admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:58.814445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1181/2395] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:39:58 2022] POST /identity/v3/projects => generated 387 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:39:58.818954 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee06c89-4651-41d5-8d1d-95d2632c41fb None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:58.831859 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee06c89-4651-41d5-8d1d-95d2632c41fb None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:58.909716 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b3379a-0a1f-47e3-8d2b-6e5a26ca40d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:58.910362 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3379a-0a1f-47e3-8d2b-6e5a26ca40d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:58.910699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3379a-0a1f-47e3-8d2b-6e5a26ca40d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:58.910974 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b3379a-0a1f-47e3-8d2b-6e5a26ca40d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:58.911724 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b3379a-0a1f-47e3-8d2b-6e5a26ca40d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:58.913785 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b3379a-0a1f-47e3-8d2b-6e5a26ca40d9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:58.999909 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1215/2396] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:39:58 2022] POST /identity/v3/users => generated 300 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:39:59.004115 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8d383-064c-4bea-a037-3502e193d7ed admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:59.018135 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8d383-064c-4bea-a037-3502e193d7ed admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:59.023604 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23f12c-8762-4a07-a288-787cf9dc85fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:59.025081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23f12c-8762-4a07-a288-787cf9dc85fe admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:59.025503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23f12c-8762-4a07-a288-787cf9dc85fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:59.025925 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce23f12c-8762-4a07-a288-787cf9dc85fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:59.026825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23f12c-8762-4a07-a288-787cf9dc85fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:59.028767 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce23f12c-8762-4a07-a288-787cf9dc85fe admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:59.044208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1182/2397] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:39:59.048729 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b3379a-0a1f-47e3-8d2b-6e5a26ca40d9 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:59.060045 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b3379a-0a1f-47e3-8d2b-6e5a26ca40d9 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:59.065383 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714a917-9732-4682-a974-4d8a0992026b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:59.066432 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714a917-9732-4682-a974-4d8a0992026b admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:59.066690 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714a917-9732-4682-a974-4d8a0992026b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:59.066966 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714a917-9732-4682-a974-4d8a0992026b admin admin] PATH_INFO: `/v3/projects/8caf1f27aa6c464d94d874c15523797b/users/cc7d3d0b25154ac99448a74740acee5f/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:59.084270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714a917-9732-4682-a974-4d8a0992026b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8caf1f27aa6c464d94d874c15523797b, user_id=cc7d3d0b25154ac99448a74740acee5f, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:59.087670 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714a917-9732-4682-a974-4d8a0992026b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:59.186201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1216/2398] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:39:59 2022] PUT /identity/v3/projects/8caf1f27aa6c464d94d874c15523797b/users/cc7d3d0b25154ac99448a74740acee5f/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:39:59.193990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23f12c-8762-4a07-a288-787cf9dc85fe admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:59.202031 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce23f12c-8762-4a07-a288-787cf9dc85fe admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:59.264842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b6c92-b1e8-4910-bf53-678c13573439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:59.265524 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b6c92-b1e8-4910-bf53-678c13573439 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:59.265830 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b6c92-b1e8-4910-bf53-678c13573439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:59.266106 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b6c92-b1e8-4910-bf53-678c13573439 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:59.266841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b6c92-b1e8-4910-bf53-678c13573439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:59.268581 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b6c92-b1e8-4910-bf53-678c13573439 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:59.279615 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1183/2399] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:39:59 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:39:59.283929 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714a917-9732-4682-a974-4d8a0992026b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:59.295136 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714a917-9732-4682-a974-4d8a0992026b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:59.299924 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948ab08-3320-4b74-b964-14d6b849e8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:59.300812 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948ab08-3320-4b74-b964-14d6b849e8bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:59.301093 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948ab08-3320-4b74-b964-14d6b849e8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:59.302603 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948ab08-3320-4b74-b964-14d6b849e8bc admin admin] PATH_INFO: `/v3/projects/8caf1f27aa6c464d94d874c15523797b/users/cc7d3d0b25154ac99448a74740acee5f/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:59.303381 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948ab08-3320-4b74-b964-14d6b849e8bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8caf1f27aa6c464d94d874c15523797b, user_id=cc7d3d0b25154ac99448a74740acee5f, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:59.306997 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948ab08-3320-4b74-b964-14d6b849e8bc admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:59.322223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1217/2400] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:39:59 2022] PUT /identity/v3/projects/8caf1f27aa6c464d94d874c15523797b/users/cc7d3d0b25154ac99448a74740acee5f/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:39:59.333853 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1a95e-5c99-445d-860a-1e7305cdf1bf None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:59.337087 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1a95e-5c99-445d-860a-1e7305cdf1bf None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:59.337087 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1a95e-5c99-445d-860a-1e7305cdf1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:59.362088 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-edc1a95e-5c99-445d-860a-1e7305cdf1bf None None] MFA Rules not processed for user `cc7d3d0b25154ac99448a74740acee5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:39:59.363313 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-edc1a95e-5c99-445d-860a-1e7305cdf1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:59.410560 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:39:59.436783 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1184/2401] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:39:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:39:59.449161 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948ab08-3320-4b74-b964-14d6b849e8bc admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:59.451963 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7948ab08-3320-4b74-b964-14d6b849e8bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:39:59.513193 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948ab08-3320-4b74-b964-14d6b849e8bc admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:59.518244 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252e28f-1d88-401f-a1a8-af796cda4e06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:59.518942 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252e28f-1d88-401f-a1a8-af796cda4e06 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:59.519289 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252e28f-1d88-401f-a1a8-af796cda4e06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:59.519608 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252e28f-1d88-401f-a1a8-af796cda4e06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:59.529362 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252e28f-1d88-401f-a1a8-af796cda4e06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:59.532521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252e28f-1d88-401f-a1a8-af796cda4e06 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:59.541181 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1218/2402] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:39:59 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:39:59.610295 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1a95e-5c99-445d-860a-1e7305cdf1bf None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:39:59.679386 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1a95e-5c99-445d-860a-1e7305cdf1bf None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:39:59.685287 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386acfa-4150-4db2-846c-e17001969625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:39:59.686430 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386acfa-4150-4db2-846c-e17001969625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:59.686708 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386acfa-4150-4db2-846c-e17001969625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:59.687027 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386acfa-4150-4db2-846c-e17001969625 admin admin] PATH_INFO: `/v3/users/a3fb1dfbde384d6987068bda8596fd95` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:59.688282 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3386acfa-4150-4db2-846c-e17001969625 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3fb1dfbde384d6987068bda8596fd95)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:39:59.690244 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3386acfa-4150-4db2-846c-e17001969625 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:39:59.859266 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3386acfa-4150-4db2-846c-e17001969625 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3fb1dfbde384d6987068bda8596fd95'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:59.859266 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3386acfa-4150-4db2-846c-e17001969625 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3fb1dfbde384d6987068bda8596fd95'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:59.932796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3386acfa-4150-4db2-846c-e17001969625 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3fb1dfbde384d6987068bda8596fd95'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:59.942203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3386acfa-4150-4db2-846c-e17001969625 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3fb1dfbde384d6987068bda8596fd95'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:59.942866 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3386acfa-4150-4db2-846c-e17001969625 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3fb1dfbde384d6987068bda8596fd95'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:39:59.962210 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1185/2403] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:39:59 2022] DELETE /identity/v3/users/a3fb1dfbde384d6987068bda8596fd95 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:39:59.966868 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f27472-5957-4170-82ab-af8471cfce5e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:39:59.967377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f27472-5957-4170-82ab-af8471cfce5e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:39:59.967521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f27472-5957-4170-82ab-af8471cfce5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:39:59.986573 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-17f27472-5957-4170-82ab-af8471cfce5e None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:39:59.987711 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-17f27472-5957-4170-82ab-af8471cfce5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:00.030240 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:40:00.053107 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1219/2404] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:39:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:40:00.063740 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386acfa-4150-4db2-846c-e17001969625 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:00.065212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3386acfa-4150-4db2-846c-e17001969625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:00.125008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386acfa-4150-4db2-846c-e17001969625 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:00.128648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec4d3d-798d-4e21-889c-99f4bb1f17f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:00.129323 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec4d3d-798d-4e21-889c-99f4bb1f17f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:00.129604 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec4d3d-798d-4e21-889c-99f4bb1f17f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:00.129882 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec4d3d-798d-4e21-889c-99f4bb1f17f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:00.139281 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fec4d3d-798d-4e21-889c-99f4bb1f17f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:00.140746 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fec4d3d-798d-4e21-889c-99f4bb1f17f0 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:00.152157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1186/2405] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:40:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:40:00.557503 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f27472-5957-4170-82ab-af8471cfce5e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:00.558934 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-17f27472-5957-4170-82ab-af8471cfce5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:00.571999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f27472-5957-4170-82ab-af8471cfce5e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:00.576748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1401e4c-4b61-40c2-8681-6c5835be8332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:00.577832 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1401e4c-4b61-40c2-8681-6c5835be8332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:00.577958 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1401e4c-4b61-40c2-8681-6c5835be8332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:00.578179 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1401e4c-4b61-40c2-8681-6c5835be8332 admin admin] PATH_INFO: `/v3/projects/0e1b3a8273d949fab03bdd973894a9d6` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:00.579388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1401e4c-4b61-40c2-8681-6c5835be8332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e1b3a8273d949fab03bdd973894a9d6)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:00.581377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1401e4c-4b61-40c2-8681-6c5835be8332 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:00.709450 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d1401e4c-4b61-40c2-8681-6c5835be8332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e1b3a8273d949fab03bdd973894a9d6', 'request_id': 'req-d1401e4c-4b61-40c2-8681-6c5835be8332'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:00.794964 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d1401e4c-4b61-40c2-8681-6c5835be8332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e1b3a8273d949fab03bdd973894a9d6', 'request_id': 'req-d1401e4c-4b61-40c2-8681-6c5835be8332'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:00.802093 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d1401e4c-4b61-40c2-8681-6c5835be8332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e1b3a8273d949fab03bdd973894a9d6', 'request_id': 'req-d1401e4c-4b61-40c2-8681-6c5835be8332'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:00.813281 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d1401e4c-4b61-40c2-8681-6c5835be8332 admin admin] The token cache is being invalidate because project 0e1b3a8273d949fab03bdd973894a9d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:40:00.813568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d1401e4c-4b61-40c2-8681-6c5835be8332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:00.814215 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-d1401e4c-4b61-40c2-8681-6c5835be8332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:00.816124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1220/2406] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:40:00 2022] DELETE /identity/v3/projects/0e1b3a8273d949fab03bdd973894a9d6 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:40:00.820097 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec4d3d-798d-4e21-889c-99f4bb1f17f0 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:00.825151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:40:00.830665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec4d3d-798d-4e21-889c-99f4bb1f17f0 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:00.892385 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e46e1b5-c472-4969-ae02-de4f495ce49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:00.893265 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46e1b5-c472-4969-ae02-de4f495ce49e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:00.893548 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46e1b5-c472-4969-ae02-de4f495ce49e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:00.893822 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e46e1b5-c472-4969-ae02-de4f495ce49e admin admin] PATH_INFO: `/v3/users/cc7d3d0b25154ac99448a74740acee5f` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:00.895039 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46e1b5-c472-4969-ae02-de4f495ce49e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc7d3d0b25154ac99448a74740acee5f)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:00.897091 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e46e1b5-c472-4969-ae02-de4f495ce49e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:01.104939 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1e46e1b5-c472-4969-ae02-de4f495ce49e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc7d3d0b25154ac99448a74740acee5f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:01.105830 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1e46e1b5-c472-4969-ae02-de4f495ce49e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc7d3d0b25154ac99448a74740acee5f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:01.155679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1e46e1b5-c472-4969-ae02-de4f495ce49e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc7d3d0b25154ac99448a74740acee5f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:01.168128 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1e46e1b5-c472-4969-ae02-de4f495ce49e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc7d3d0b25154ac99448a74740acee5f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:01.168987 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-1e46e1b5-c472-4969-ae02-de4f495ce49e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc7d3d0b25154ac99448a74740acee5f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:01.188174 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1187/2407] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:40:00 2022] DELETE /identity/v3/users/cc7d3d0b25154ac99448a74740acee5f => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:40:01.665364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1401e4c-4b61-40c2-8681-6c5835be8332 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:01.667346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d1401e4c-4b61-40c2-8681-6c5835be8332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:01.682011 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1401e4c-4b61-40c2-8681-6c5835be8332 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:01.741247 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd0ea20-a708-4fd5-873a-388c9cfea6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:01.742163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0ea20-a708-4fd5-873a-388c9cfea6dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:01.742455 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0ea20-a708-4fd5-873a-388c9cfea6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:01.742736 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd0ea20-a708-4fd5-873a-388c9cfea6dc admin admin] PATH_INFO: `/v3/projects/8caf1f27aa6c464d94d874c15523797b` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:01.744004 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0ea20-a708-4fd5-873a-388c9cfea6dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8caf1f27aa6c464d94d874c15523797b)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:01.746009 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd0ea20-a708-4fd5-873a-388c9cfea6dc admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:01.894373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9fd0ea20-a708-4fd5-873a-388c9cfea6dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8caf1f27aa6c464d94d874c15523797b', 'request_id': 'req-9fd0ea20-a708-4fd5-873a-388c9cfea6dc'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:01.966446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9fd0ea20-a708-4fd5-873a-388c9cfea6dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8caf1f27aa6c464d94d874c15523797b', 'request_id': 'req-9fd0ea20-a708-4fd5-873a-388c9cfea6dc'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:01.973553 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9fd0ea20-a708-4fd5-873a-388c9cfea6dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8caf1f27aa6c464d94d874c15523797b', 'request_id': 'req-9fd0ea20-a708-4fd5-873a-388c9cfea6dc'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:01.978422 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:40:01.984650 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9fd0ea20-a708-4fd5-873a-388c9cfea6dc admin admin] The token cache is being invalidate because project 8caf1f27aa6c464d94d874c15523797b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:40:01.984946 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9fd0ea20-a708-4fd5-873a-388c9cfea6dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:01.985591 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-9fd0ea20-a708-4fd5-873a-388c9cfea6dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:01.987145 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1221/2408] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:40:01 2022] DELETE /identity/v3/projects/8caf1f27aa6c464d94d874c15523797b => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:40:01.992235 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5302c-ef60-4212-b1d2-ef3a641fcb73 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:01.992756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5302c-ef60-4212-b1d2-ef3a641fcb73 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:01.992869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c5302c-ef60-4212-b1d2-ef3a641fcb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:02.013771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-78c5302c-ef60-4212-b1d2-ef3a641fcb73 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:40:02.015029 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-78c5302c-ef60-4212-b1d2-ef3a641fcb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:02.075867 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1188/2409] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:40:02.083785 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1ad90-5ec4-4657-b965-148fd9ca517f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:02.084078 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1ad90-5ec4-4657-b965-148fd9ca517f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:02.084367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f1ad90-5ec4-4657-b965-148fd9ca517f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:02.103761 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-34f1ad90-5ec4-4657-b965-148fd9ca517f None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:40:02.105030 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-34f1ad90-5ec4-4657-b965-148fd9ca517f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:02.113295 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1222/2410] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:40:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:40:02.117827 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5302c-ef60-4212-b1d2-ef3a641fcb73 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:02.128130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c5302c-ef60-4212-b1d2-ef3a641fcb73 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:02.134386 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99f24a-b062-4ed3-aef4-8ff6272db566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:02.135225 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99f24a-b062-4ed3-aef4-8ff6272db566 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:02.135529 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99f24a-b062-4ed3-aef4-8ff6272db566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:02.135803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99f24a-b062-4ed3-aef4-8ff6272db566 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:02.136537 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c99f24a-b062-4ed3-aef4-8ff6272db566 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:40:02.136845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c99f24a-b062-4ed3-aef4-8ff6272db566 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:02.138521 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c99f24a-b062-4ed3-aef4-8ff6272db566 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:02.152095 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1189/2411] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:40:02.156626 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1ad90-5ec4-4657-b965-148fd9ca517f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:02.166775 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f1ad90-5ec4-4657-b965-148fd9ca517f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:02.171535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249738ab-9438-475e-a6fd-56c92f18b736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:02.172189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249738ab-9438-475e-a6fd-56c92f18b736 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:02.172467 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249738ab-9438-475e-a6fd-56c92f18b736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:02.172737 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249738ab-9438-475e-a6fd-56c92f18b736 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:02.173464 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249738ab-9438-475e-a6fd-56c92f18b736 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:02.175374 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249738ab-9438-475e-a6fd-56c92f18b736 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:02.260646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1223/2412] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:40:02 2022] POST /identity/v3/projects => generated 409 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:40:02.266124 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99f24a-b062-4ed3-aef4-8ff6272db566 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:02.276352 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99f24a-b062-4ed3-aef4-8ff6272db566 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:02.281490 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:40:02.351934 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302646f-bd47-4bab-9130-fdbf389898b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:02.352836 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302646f-bd47-4bab-9130-fdbf389898b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:02.353327 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302646f-bd47-4bab-9130-fdbf389898b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:02.354391 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302646f-bd47-4bab-9130-fdbf389898b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:02.355595 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302646f-bd47-4bab-9130-fdbf389898b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:02.359198 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302646f-bd47-4bab-9130-fdbf389898b7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:02.429989 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1190/2413] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:40:02 2022] POST /identity/v3/users => generated 305 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:40:02.435143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249738ab-9438-475e-a6fd-56c92f18b736 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:02.446737 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249738ab-9438-475e-a6fd-56c92f18b736 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:02.451833 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994a6ee-2c0e-4bc3-8149-76f540308af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:02.452509 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994a6ee-2c0e-4bc3-8149-76f540308af7 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:02.452791 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994a6ee-2c0e-4bc3-8149-76f540308af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:02.453065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994a6ee-2c0e-4bc3-8149-76f540308af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:02.453781 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994a6ee-2c0e-4bc3-8149-76f540308af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:02.455501 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994a6ee-2c0e-4bc3-8149-76f540308af7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:02.466760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1224/2414] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:40:02.471011 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302646f-bd47-4bab-9130-fdbf389898b7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:02.481312 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302646f-bd47-4bab-9130-fdbf389898b7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:02.485934 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba263b4-b961-4202-8720-ee52624d2bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:02.486876 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba263b4-b961-4202-8720-ee52624d2bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:02.487162 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba263b4-b961-4202-8720-ee52624d2bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:02.494635 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba263b4-b961-4202-8720-ee52624d2bb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:02.494635 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba263b4-b961-4202-8720-ee52624d2bb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:02.494635 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba263b4-b961-4202-8720-ee52624d2bb8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:02.500937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1191/2415] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:40:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:40:02.506161 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994a6ee-2c0e-4bc3-8149-76f540308af7 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:02.517334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994a6ee-2c0e-4bc3-8149-76f540308af7 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:02.522022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ab9a97-a912-4529-99d7-e5fabb982b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:02.522905 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab9a97-a912-4529-99d7-e5fabb982b3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:02.523190 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab9a97-a912-4529-99d7-e5fabb982b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:02.523494 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab9a97-a912-4529-99d7-e5fabb982b3b admin admin] PATH_INFO: `/v3/projects/e80880d660a34a20b9a540a396eab77d/users/21fd094b72894520b206f3d0a9d0813a/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:02.538434 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ab9a97-a912-4529-99d7-e5fabb982b3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e80880d660a34a20b9a540a396eab77d, user_id=21fd094b72894520b206f3d0a9d0813a, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:02.541198 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ab9a97-a912-4529-99d7-e5fabb982b3b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:02.623340 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1225/2416] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:40:02 2022] PUT /identity/v3/projects/e80880d660a34a20b9a540a396eab77d/users/21fd094b72894520b206f3d0a9d0813a/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:40:02.634019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1616e-308b-4c23-95c6-13fd2598e513 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:02.634313 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1616e-308b-4c23-95c6-13fd2598e513 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:02.634598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1616e-308b-4c23-95c6-13fd2598e513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:02.663148 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-09e1616e-308b-4c23-95c6-13fd2598e513 None None] MFA Rules not processed for user `21fd094b72894520b206f3d0a9d0813a`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:40:02.664402 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-09e1616e-308b-4c23-95c6-13fd2598e513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:02.731668 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1192/2417] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:40:02 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:40:02.736210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ab9a97-a912-4529-99d7-e5fabb982b3b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:02.804393 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ab9a97-a912-4529-99d7-e5fabb982b3b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:02.809273 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41913479-34a8-489b-bc42-b4f5904381d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:02.809971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41913479-34a8-489b-bc42-b4f5904381d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:02.810253 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41913479-34a8-489b-bc42-b4f5904381d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:02.810531 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41913479-34a8-489b-bc42-b4f5904381d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:02.811572 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41913479-34a8-489b-bc42-b4f5904381d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:02.813557 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41913479-34a8-489b-bc42-b4f5904381d0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:02.973944 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1226/2418] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:40:02 2022] POST /identity/v3/projects => generated 409 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:40:02.978612 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1616e-308b-4c23-95c6-13fd2598e513 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:02.989779 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1616e-308b-4c23-95c6-13fd2598e513 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:03.052490 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b678cf-d906-4074-9aaa-9fc23f3957dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:03.053401 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b678cf-d906-4074-9aaa-9fc23f3957dc admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:03.053690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b678cf-d906-4074-9aaa-9fc23f3957dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:03.054017 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b678cf-d906-4074-9aaa-9fc23f3957dc admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:03.054821 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b678cf-d906-4074-9aaa-9fc23f3957dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:03.057323 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b678cf-d906-4074-9aaa-9fc23f3957dc admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:03.144333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1193/2419] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:40:02 2022] POST /identity/v3/users => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:40:03.149064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41913479-34a8-489b-bc42-b4f5904381d0 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:03.159718 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41913479-34a8-489b-bc42-b4f5904381d0 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:03.164351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb02146-ecbf-4466-b66e-a43a1919035c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:03.165061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb02146-ecbf-4466-b66e-a43a1919035c admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:03.165416 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb02146-ecbf-4466-b66e-a43a1919035c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:03.165725 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb02146-ecbf-4466-b66e-a43a1919035c admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:03.166466 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb02146-ecbf-4466-b66e-a43a1919035c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:03.168109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb02146-ecbf-4466-b66e-a43a1919035c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:03.179327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1227/2420] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:40:03.184204 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b678cf-d906-4074-9aaa-9fc23f3957dc admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:03.194589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b678cf-d906-4074-9aaa-9fc23f3957dc admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:03.200070 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ef4ff-9f66-4770-9ad5-c77502424cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:03.200987 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ef4ff-9f66-4770-9ad5-c77502424cb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:03.201264 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ef4ff-9f66-4770-9ad5-c77502424cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:03.201538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03ef4ff-9f66-4770-9ad5-c77502424cb1 admin admin] PATH_INFO: `/v3/projects/d70732fdd9d3464e8191386d28f9f7f3/users/09467b3a7d8a4954ae69b5b7ad0f8b65/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:03.203909 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:40:03.217564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03ef4ff-9f66-4770-9ad5-c77502424cb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d70732fdd9d3464e8191386d28f9f7f3, user_id=09467b3a7d8a4954ae69b5b7ad0f8b65, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:03.221494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03ef4ff-9f66-4770-9ad5-c77502424cb1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:03.328312 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1194/2421] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:40:03 2022] PUT /identity/v3/projects/d70732fdd9d3464e8191386d28f9f7f3/users/09467b3a7d8a4954ae69b5b7ad0f8b65/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:40:03.332421 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb02146-ecbf-4466-b66e-a43a1919035c admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:03.342566 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb02146-ecbf-4466-b66e-a43a1919035c admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:03.402778 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18afad8c-abe2-4686-89d1-3f110b988336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:03.403453 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18afad8c-abe2-4686-89d1-3f110b988336 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:03.403785 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18afad8c-abe2-4686-89d1-3f110b988336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:03.404054 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18afad8c-abe2-4686-89d1-3f110b988336 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:03.404787 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18afad8c-abe2-4686-89d1-3f110b988336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:03.406777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18afad8c-abe2-4686-89d1-3f110b988336 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:03.418845 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1228/2422] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:40:03 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:40:03.423152 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ef4ff-9f66-4770-9ad5-c77502424cb1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:03.434850 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03ef4ff-9f66-4770-9ad5-c77502424cb1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:03.440128 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7d3f2-fb07-46bf-8a73-28fddbe59ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:03.441028 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7d3f2-fb07-46bf-8a73-28fddbe59ced admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:03.441319 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7d3f2-fb07-46bf-8a73-28fddbe59ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:03.441600 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7d3f2-fb07-46bf-8a73-28fddbe59ced admin admin] PATH_INFO: `/v3/projects/d70732fdd9d3464e8191386d28f9f7f3/users/09467b3a7d8a4954ae69b5b7ad0f8b65/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:03.443629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7d3f2-fb07-46bf-8a73-28fddbe59ced admin admin] RBAC: Authorizing `identity:create_grant(project_id=d70732fdd9d3464e8191386d28f9f7f3, user_id=09467b3a7d8a4954ae69b5b7ad0f8b65, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:03.446339 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7d3f2-fb07-46bf-8a73-28fddbe59ced admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:03.457762 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1195/2423] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:40:03 2022] PUT /identity/v3/projects/d70732fdd9d3464e8191386d28f9f7f3/users/09467b3a7d8a4954ae69b5b7ad0f8b65/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:40:03.469331 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669a8cb-7aad-4b50-9bea-beffce4d4155 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:03.469624 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669a8cb-7aad-4b50-9bea-beffce4d4155 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:03.469908 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669a8cb-7aad-4b50-9bea-beffce4d4155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:03.476087 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:40:03.499913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8669a8cb-7aad-4b50-9bea-beffce4d4155 None None] MFA Rules not processed for user `09467b3a7d8a4954ae69b5b7ad0f8b65`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:40:03.501121 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8669a8cb-7aad-4b50-9bea-beffce4d4155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:03.576513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1229/2424] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:40:03 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:40:03.587207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7d3f2-fb07-46bf-8a73-28fddbe59ced admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:03.589506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a7a7d3f2-fb07-46bf-8a73-28fddbe59ced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:03.648438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7d3f2-fb07-46bf-8a73-28fddbe59ced admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:03.652236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ebc2f8-93cd-43c2-9c1a-0ac65000fc3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:03.652927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ebc2f8-93cd-43c2-9c1a-0ac65000fc3f service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:03.653215 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ebc2f8-93cd-43c2-9c1a-0ac65000fc3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:03.653495 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ebc2f8-93cd-43c2-9c1a-0ac65000fc3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:03.663432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ebc2f8-93cd-43c2-9c1a-0ac65000fc3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:03.664896 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ebc2f8-93cd-43c2-9c1a-0ac65000fc3f service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:03.675361 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1196/2425] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:40:03 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:40:04.021651 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669a8cb-7aad-4b50-9bea-beffce4d4155 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:04.032203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669a8cb-7aad-4b50-9bea-beffce4d4155 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:04.035723 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43cf7e-4356-47a8-a99a-ee03a89e4ea9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:04.036360 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43cf7e-4356-47a8-a99a-ee03a89e4ea9 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:04.036642 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43cf7e-4356-47a8-a99a-ee03a89e4ea9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:04.036920 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43cf7e-4356-47a8-a99a-ee03a89e4ea9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:04.095104 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce43cf7e-4356-47a8-a99a-ee03a89e4ea9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:04.096592 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce43cf7e-4356-47a8-a99a-ee03a89e4ea9 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:04.106541 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1230/2426] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:40:04 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:40:13.670970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=38,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:40:14.101785 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=61 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:40:14.327813 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ebc2f8-93cd-43c2-9c1a-0ac65000fc3f service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:14.396646 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ebc2f8-93cd-43c2-9c1a-0ac65000fc3f service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:14.401199 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dadf62-c083-40ad-9f7c-f0da30b752ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:14.402085 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dadf62-c083-40ad-9f7c-f0da30b752ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:14.402388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dadf62-c083-40ad-9f7c-f0da30b752ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:14.402647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dadf62-c083-40ad-9f7c-f0da30b752ac admin admin] PATH_INFO: `/v3/users/21fd094b72894520b206f3d0a9d0813a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:14.404081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dadf62-c083-40ad-9f7c-f0da30b752ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=21fd094b72894520b206f3d0a9d0813a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:14.406300 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dadf62-c083-40ad-9f7c-f0da30b752ac admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:14.636094 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-64dadf62-c083-40ad-9f7c-f0da30b752ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21fd094b72894520b206f3d0a9d0813a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:14.637154 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-64dadf62-c083-40ad-9f7c-f0da30b752ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21fd094b72894520b206f3d0a9d0813a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:14.698667 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-64dadf62-c083-40ad-9f7c-f0da30b752ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21fd094b72894520b206f3d0a9d0813a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:14.709576 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-64dadf62-c083-40ad-9f7c-f0da30b752ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21fd094b72894520b206f3d0a9d0813a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:14.710563 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-64dadf62-c083-40ad-9f7c-f0da30b752ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21fd094b72894520b206f3d0a9d0813a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:14.729819 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1197/2427] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:40:14 2022] DELETE /identity/v3/users/21fd094b72894520b206f3d0a9d0813a => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:40:14.735017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b484f-0cfb-45d0-9f8f-3134f1fb97d0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:14.735311 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b484f-0cfb-45d0-9f8f-3134f1fb97d0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:14.735640 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b484f-0cfb-45d0-9f8f-3134f1fb97d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:14.755150 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-bf9b484f-0cfb-45d0-9f8f-3134f1fb97d0 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:40:14.756398 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bf9b484f-0cfb-45d0-9f8f-3134f1fb97d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:14.817647 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1231/2428] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:40:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:40:14.827654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dadf62-c083-40ad-9f7c-f0da30b752ac admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:14.829720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-64dadf62-c083-40ad-9f7c-f0da30b752ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:14.890982 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dadf62-c083-40ad-9f7c-f0da30b752ac admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:14.894700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54b316-9e6c-47e4-a397-bb00e7b561ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:14.895359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54b316-9e6c-47e4-a397-bb00e7b561ef service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:14.895670 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54b316-9e6c-47e4-a397-bb00e7b561ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:14.895946 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c54b316-9e6c-47e4-a397-bb00e7b561ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:14.906394 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54b316-9e6c-47e4-a397-bb00e7b561ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:14.907856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c54b316-9e6c-47e4-a397-bb00e7b561ef service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:14.917626 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1198/2429] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:40:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:40:15.357950 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b484f-0cfb-45d0-9f8f-3134f1fb97d0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:15.360268 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bf9b484f-0cfb-45d0-9f8f-3134f1fb97d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:15.374611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b484f-0cfb-45d0-9f8f-3134f1fb97d0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:15.379595 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a75a06c-3561-46cb-b45d-acaf909b10ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:15.380473 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75a06c-3561-46cb-b45d-acaf909b10ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:15.380749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75a06c-3561-46cb-b45d-acaf909b10ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:15.381021 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75a06c-3561-46cb-b45d-acaf909b10ed admin admin] PATH_INFO: `/v3/projects/e80880d660a34a20b9a540a396eab77d` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:15.382250 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a75a06c-3561-46cb-b45d-acaf909b10ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=e80880d660a34a20b9a540a396eab77d)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:15.384250 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a75a06c-3561-46cb-b45d-acaf909b10ed admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:15.538276 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5a75a06c-3561-46cb-b45d-acaf909b10ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e80880d660a34a20b9a540a396eab77d', 'request_id': 'req-5a75a06c-3561-46cb-b45d-acaf909b10ed'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:15.590152 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5a75a06c-3561-46cb-b45d-acaf909b10ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e80880d660a34a20b9a540a396eab77d', 'request_id': 'req-5a75a06c-3561-46cb-b45d-acaf909b10ed'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:15.597428 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5a75a06c-3561-46cb-b45d-acaf909b10ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e80880d660a34a20b9a540a396eab77d', 'request_id': 'req-5a75a06c-3561-46cb-b45d-acaf909b10ed'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:15.609481 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5a75a06c-3561-46cb-b45d-acaf909b10ed admin admin] The token cache is being invalidate because project e80880d660a34a20b9a540a396eab77d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:40:15.609790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5a75a06c-3561-46cb-b45d-acaf909b10ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:15.610360 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5a75a06c-3561-46cb-b45d-acaf909b10ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:15.612225 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1232/2430] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:40:15 2022] DELETE /identity/v3/projects/e80880d660a34a20b9a540a396eab77d => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:40:15.616189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54b316-9e6c-47e4-a397-bb00e7b561ef service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:15.627022 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c54b316-9e6c-47e4-a397-bb00e7b561ef service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:15.646909 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:40:15.686383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45b9b17-1a21-4df3-ad7e-bc495feef62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:15.687998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b9b17-1a21-4df3-ad7e-bc495feef62d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:15.688276 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b9b17-1a21-4df3-ad7e-bc495feef62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:15.688554 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45b9b17-1a21-4df3-ad7e-bc495feef62d admin admin] PATH_INFO: `/v3/users/09467b3a7d8a4954ae69b5b7ad0f8b65` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:15.689743 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b9b17-1a21-4df3-ad7e-bc495feef62d admin admin] RBAC: Authorizing `identity:delete_user(user_id=09467b3a7d8a4954ae69b5b7ad0f8b65)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:15.691760 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45b9b17-1a21-4df3-ad7e-bc495feef62d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:15.867131 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d45b9b17-1a21-4df3-ad7e-bc495feef62d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09467b3a7d8a4954ae69b5b7ad0f8b65'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:15.867985 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d45b9b17-1a21-4df3-ad7e-bc495feef62d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09467b3a7d8a4954ae69b5b7ad0f8b65'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:15.955734 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d45b9b17-1a21-4df3-ad7e-bc495feef62d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09467b3a7d8a4954ae69b5b7ad0f8b65'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:15.963279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d45b9b17-1a21-4df3-ad7e-bc495feef62d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09467b3a7d8a4954ae69b5b7ad0f8b65'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:15.964029 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d45b9b17-1a21-4df3-ad7e-bc495feef62d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09467b3a7d8a4954ae69b5b7ad0f8b65'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:15.981649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1199/2431] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:40:15 2022] DELETE /identity/v3/users/09467b3a7d8a4954ae69b5b7ad0f8b65 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:40:16.449119 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a75a06c-3561-46cb-b45d-acaf909b10ed admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:16.450797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5a75a06c-3561-46cb-b45d-acaf909b10ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:16.469843 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a75a06c-3561-46cb-b45d-acaf909b10ed admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:16.534076 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cd1580-345c-4751-b912-19d1d256883d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:16.534951 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd1580-345c-4751-b912-19d1d256883d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:16.535227 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd1580-345c-4751-b912-19d1d256883d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:16.535525 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cd1580-345c-4751-b912-19d1d256883d admin admin] PATH_INFO: `/v3/projects/d70732fdd9d3464e8191386d28f9f7f3` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:16.536817 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd1580-345c-4751-b912-19d1d256883d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d70732fdd9d3464e8191386d28f9f7f3)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:16.538776 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cd1580-345c-4751-b912-19d1d256883d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:16.671416 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:40:16.690645 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f1cd1580-345c-4751-b912-19d1d256883d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd70732fdd9d3464e8191386d28f9f7f3', 'request_id': 'req-f1cd1580-345c-4751-b912-19d1d256883d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:16.750835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f1cd1580-345c-4751-b912-19d1d256883d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd70732fdd9d3464e8191386d28f9f7f3', 'request_id': 'req-f1cd1580-345c-4751-b912-19d1d256883d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:16.758832 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f1cd1580-345c-4751-b912-19d1d256883d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd70732fdd9d3464e8191386d28f9f7f3', 'request_id': 'req-f1cd1580-345c-4751-b912-19d1d256883d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:16.770670 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f1cd1580-345c-4751-b912-19d1d256883d admin admin] The token cache is being invalidate because project d70732fdd9d3464e8191386d28f9f7f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:40:16.770986 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f1cd1580-345c-4751-b912-19d1d256883d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:16.771670 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f1cd1580-345c-4751-b912-19d1d256883d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:16.773509 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1233/2432] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:40:16 2022] DELETE /identity/v3/projects/d70732fdd9d3464e8191386d28f9f7f3 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:40:16.778299 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd6609-c3b5-41e3-9567-7c48be9ae146 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:16.778561 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd6609-c3b5-41e3-9567-7c48be9ae146 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:16.778825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd6609-c3b5-41e3-9567-7c48be9ae146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:16.798284 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-83bd6609-c3b5-41e3-9567-7c48be9ae146 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:40:16.799641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-83bd6609-c3b5-41e3-9567-7c48be9ae146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:16.867293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1200/2433] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:40:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:40:16.876041 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd015b1-adb8-428f-849b-b5960c3916d2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:16.876331 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd015b1-adb8-428f-849b-b5960c3916d2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:16.876613 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd015b1-adb8-428f-849b-b5960c3916d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:16.896026 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5cd015b1-adb8-428f-849b-b5960c3916d2 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:40:16.897410 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5cd015b1-adb8-428f-849b-b5960c3916d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:16.905952 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1234/2434] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:40:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:40:16.910546 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd6609-c3b5-41e3-9567-7c48be9ae146 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:16.921146 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd6609-c3b5-41e3-9567-7c48be9ae146 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:16.926741 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28162acf-1089-44a0-b43a-f329beb36425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:16.927605 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28162acf-1089-44a0-b43a-f329beb36425 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:16.927896 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28162acf-1089-44a0-b43a-f329beb36425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:16.928174 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28162acf-1089-44a0-b43a-f329beb36425 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:16.928938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28162acf-1089-44a0-b43a-f329beb36425 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:40:16.929250 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28162acf-1089-44a0-b43a-f329beb36425 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:16.930920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28162acf-1089-44a0-b43a-f329beb36425 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:16.944128 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1201/2435] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:40:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:40:16.948770 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd015b1-adb8-428f-849b-b5960c3916d2 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:16.958229 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd015b1-adb8-428f-849b-b5960c3916d2 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:16.965089 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67779cb6-3f6e-436a-9c54-c4d18eca1e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:16.965089 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67779cb6-3f6e-436a-9c54-c4d18eca1e16 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:16.965089 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67779cb6-3f6e-436a-9c54-c4d18eca1e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:16.965089 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67779cb6-3f6e-436a-9c54-c4d18eca1e16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:16.965498 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67779cb6-3f6e-436a-9c54-c4d18eca1e16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:16.966971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67779cb6-3f6e-436a-9c54-c4d18eca1e16 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:17.090250 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1235/2436] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:40:16 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:40:17.094947 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28162acf-1089-44a0-b43a-f329beb36425 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:17.105516 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28162acf-1089-44a0-b43a-f329beb36425 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:17.133835 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:40:17.165784 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4c00e-a2fa-45d6-bc3f-5def58dd5e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:17.166463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4c00e-a2fa-45d6-bc3f-5def58dd5e97 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:17.166747 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4c00e-a2fa-45d6-bc3f-5def58dd5e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:17.167168 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4c00e-a2fa-45d6-bc3f-5def58dd5e97 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:17.167912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d4c00e-a2fa-45d6-bc3f-5def58dd5e97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:17.175354 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d4c00e-a2fa-45d6-bc3f-5def58dd5e97 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:17.281862 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1202/2437] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:40:17 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:40:17.286440 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67779cb6-3f6e-436a-9c54-c4d18eca1e16 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:17.297886 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67779cb6-3f6e-436a-9c54-c4d18eca1e16 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:17.304419 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abcbd8f-5688-43b5-9183-83fbf6d955e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:17.305753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcbd8f-5688-43b5-9183-83fbf6d955e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:17.306042 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcbd8f-5688-43b5-9183-83fbf6d955e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:17.306321 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcbd8f-5688-43b5-9183-83fbf6d955e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:17.308932 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abcbd8f-5688-43b5-9183-83fbf6d955e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:17.310622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abcbd8f-5688-43b5-9183-83fbf6d955e1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:17.328025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1236/2438] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:40:17.334230 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4c00e-a2fa-45d6-bc3f-5def58dd5e97 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:17.344851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4c00e-a2fa-45d6-bc3f-5def58dd5e97 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:17.349881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac9727a-38eb-4acd-88b8-0f94f4db6c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:17.350560 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9727a-38eb-4acd-88b8-0f94f4db6c3b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:17.351420 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9727a-38eb-4acd-88b8-0f94f4db6c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:17.351757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac9727a-38eb-4acd-88b8-0f94f4db6c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:17.352505 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac9727a-38eb-4acd-88b8-0f94f4db6c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:17.354192 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac9727a-38eb-4acd-88b8-0f94f4db6c3b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:17.365381 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1203/2439] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:40:17.369358 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abcbd8f-5688-43b5-9183-83fbf6d955e1 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:17.380040 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abcbd8f-5688-43b5-9183-83fbf6d955e1 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:17.390148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba808854-8625-4e0c-8b72-3dfdf60118c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:17.391737 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba808854-8625-4e0c-8b72-3dfdf60118c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:17.392021 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba808854-8625-4e0c-8b72-3dfdf60118c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:17.392304 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba808854-8625-4e0c-8b72-3dfdf60118c7 admin admin] PATH_INFO: `/v3/projects/9469bcd48a124cdf9c4c5e50ad1abcc9/users/895345c685304bbba6088e98b3c3174d/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:17.407852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba808854-8625-4e0c-8b72-3dfdf60118c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9469bcd48a124cdf9c4c5e50ad1abcc9, user_id=895345c685304bbba6088e98b3c3174d, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:17.410525 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba808854-8625-4e0c-8b72-3dfdf60118c7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:17.482133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1237/2440] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:40:17 2022] PUT /identity/v3/projects/9469bcd48a124cdf9c4c5e50ad1abcc9/users/895345c685304bbba6088e98b3c3174d/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:40:17.493298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6bc60-2951-420f-b008-1b8f1b385d4f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:17.493585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6bc60-2951-420f-b008-1b8f1b385d4f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:17.493863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d6bc60-2951-420f-b008-1b8f1b385d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:17.520769 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e0d6bc60-2951-420f-b008-1b8f1b385d4f None None] MFA Rules not processed for user `895345c685304bbba6088e98b3c3174d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:40:17.521981 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e0d6bc60-2951-420f-b008-1b8f1b385d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:17.586646 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1204/2441] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:40:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:40:17.591257 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba808854-8625-4e0c-8b72-3dfdf60118c7 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:17.653243 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba808854-8625-4e0c-8b72-3dfdf60118c7 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:17.658460 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20d9e3-2cfe-472e-8406-a056f453824f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:17.659108 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20d9e3-2cfe-472e-8406-a056f453824f admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:17.659390 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20d9e3-2cfe-472e-8406-a056f453824f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:17.659694 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20d9e3-2cfe-472e-8406-a056f453824f admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:17.660385 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf20d9e3-2cfe-472e-8406-a056f453824f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:17.662282 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf20d9e3-2cfe-472e-8406-a056f453824f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:17.752611 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1238/2442] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:40:17 2022] POST /identity/v3/projects => generated 383 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:40:17.757015 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d6bc60-2951-420f-b008-1b8f1b385d4f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:17.768454 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d6bc60-2951-420f-b008-1b8f1b385d4f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:17.826244 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb228ec-1243-4c54-bdc4-7a0d1cabeb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:17.826910 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb228ec-1243-4c54-bdc4-7a0d1cabeb92 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:17.827192 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb228ec-1243-4c54-bdc4-7a0d1cabeb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:17.827465 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb228ec-1243-4c54-bdc4-7a0d1cabeb92 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:17.828917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb228ec-1243-4c54-bdc4-7a0d1cabeb92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:17.831113 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb228ec-1243-4c54-bdc4-7a0d1cabeb92 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:17.927837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1205/2443] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:40:17 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:40:17.931834 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20d9e3-2cfe-472e-8406-a056f453824f admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:17.944080 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20d9e3-2cfe-472e-8406-a056f453824f admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:17.948925 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054a65d-e956-4785-af8e-7b24eeb50159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:17.949594 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054a65d-e956-4785-af8e-7b24eeb50159 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:17.949880 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054a65d-e956-4785-af8e-7b24eeb50159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:17.950162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054a65d-e956-4785-af8e-7b24eeb50159 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:17.950896 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054a65d-e956-4785-af8e-7b24eeb50159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:17.952544 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054a65d-e956-4785-af8e-7b24eeb50159 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:17.964766 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1239/2444] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:40:17.968712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb228ec-1243-4c54-bdc4-7a0d1cabeb92 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:17.979254 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb228ec-1243-4c54-bdc4-7a0d1cabeb92 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:17.984873 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307f401-baf6-435e-afdc-47a3e7b33e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:17.985204 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307f401-baf6-435e-afdc-47a3e7b33e90 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:17.985539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307f401-baf6-435e-afdc-47a3e7b33e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:17.985822 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307f401-baf6-435e-afdc-47a3e7b33e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:17.986553 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307f401-baf6-435e-afdc-47a3e7b33e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:17.988187 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307f401-baf6-435e-afdc-47a3e7b33e90 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:17.999199 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1206/2445] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:40:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:40:18.007616 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054a65d-e956-4785-af8e-7b24eeb50159 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:18.019876 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054a65d-e956-4785-af8e-7b24eeb50159 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:18.025183 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ce7ba-b611-4b48-9249-e9b5b4fdef44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:18.026080 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ce7ba-b611-4b48-9249-e9b5b4fdef44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:18.026368 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ce7ba-b611-4b48-9249-e9b5b4fdef44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:18.026641 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ce7ba-b611-4b48-9249-e9b5b4fdef44 admin admin] PATH_INFO: `/v3/projects/7f70f01ea36744929f202f016184e71f/users/a96a4ddbd69b446da0f73de51cb1e2a3/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:18.041471 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ce7ba-b611-4b48-9249-e9b5b4fdef44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f70f01ea36744929f202f016184e71f, user_id=a96a4ddbd69b446da0f73de51cb1e2a3, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:18.044194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ce7ba-b611-4b48-9249-e9b5b4fdef44 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:18.151815 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:40:18.156702 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1240/2446] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:40:18 2022] PUT /identity/v3/projects/7f70f01ea36744929f202f016184e71f/users/a96a4ddbd69b446da0f73de51cb1e2a3/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:40:18.229092 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83955aa4-eae0-44c9-b285-9e1ae821c56d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:18.229392 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83955aa4-eae0-44c9-b285-9e1ae821c56d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:18.229727 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83955aa4-eae0-44c9-b285-9e1ae821c56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:18.259117 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-83955aa4-eae0-44c9-b285-9e1ae821c56d None None] MFA Rules not processed for user `a96a4ddbd69b446da0f73de51cb1e2a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:40:18.260414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-83955aa4-eae0-44c9-b285-9e1ae821c56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:18.264746 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:40:18.329211 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1207/2447] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:40:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:40:18.333875 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ce7ba-b611-4b48-9249-e9b5b4fdef44 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:18.399672 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ce7ba-b611-4b48-9249-e9b5b4fdef44 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:18.404529 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0019fb54-3463-4c52-8487-b1c1af82939e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:18.405120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019fb54-3463-4c52-8487-b1c1af82939e admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:18.405407 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019fb54-3463-4c52-8487-b1c1af82939e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:18.405687 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019fb54-3463-4c52-8487-b1c1af82939e admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:18.406460 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0019fb54-3463-4c52-8487-b1c1af82939e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:18.408437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0019fb54-3463-4c52-8487-b1c1af82939e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:18.511925 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1241/2448] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:40:18 2022] POST /identity/v3/projects => generated 385 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:40:18.516485 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83955aa4-eae0-44c9-b285-9e1ae821c56d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:18.527609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83955aa4-eae0-44c9-b285-9e1ae821c56d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:18.590391 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518125e-ce99-4b3b-a7ca-75e00609ed4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:18.591060 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518125e-ce99-4b3b-a7ca-75e00609ed4a admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:18.591344 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518125e-ce99-4b3b-a7ca-75e00609ed4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:18.591657 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518125e-ce99-4b3b-a7ca-75e00609ed4a admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:18.592362 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518125e-ce99-4b3b-a7ca-75e00609ed4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:18.594919 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518125e-ce99-4b3b-a7ca-75e00609ed4a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:18.688946 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1208/2449] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:40:18 2022] POST /identity/v3/users => generated 299 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:40:18.692957 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0019fb54-3463-4c52-8487-b1c1af82939e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:18.703289 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0019fb54-3463-4c52-8487-b1c1af82939e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:18.708017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245c8a2-8c9a-470a-aed5-8cde8ccfbc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:18.708670 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245c8a2-8c9a-470a-aed5-8cde8ccfbc3d admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:18.708952 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245c8a2-8c9a-470a-aed5-8cde8ccfbc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:18.709230 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7245c8a2-8c9a-470a-aed5-8cde8ccfbc3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:18.709950 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245c8a2-8c9a-470a-aed5-8cde8ccfbc3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:18.711597 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7245c8a2-8c9a-470a-aed5-8cde8ccfbc3d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:18.723590 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1242/2450] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:40:18.728720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518125e-ce99-4b3b-a7ca-75e00609ed4a admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:18.740066 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518125e-ce99-4b3b-a7ca-75e00609ed4a admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:18.745377 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec0f6a-a54a-4fbe-8a3e-55f8afcbe457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:18.746280 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec0f6a-a54a-4fbe-8a3e-55f8afcbe457 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:18.746562 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec0f6a-a54a-4fbe-8a3e-55f8afcbe457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:18.746844 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec0f6a-a54a-4fbe-8a3e-55f8afcbe457 admin admin] PATH_INFO: `/v3/projects/a7b751318a744aeb91642dde57ca620b/users/a069c3de26ef47e9b581461d317dc28e/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:18.764562 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec0f6a-a54a-4fbe-8a3e-55f8afcbe457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7b751318a744aeb91642dde57ca620b, user_id=a069c3de26ef47e9b581461d317dc28e, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:18.767301 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec0f6a-a54a-4fbe-8a3e-55f8afcbe457 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:18.877636 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1209/2451] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:40:18 2022] PUT /identity/v3/projects/a7b751318a744aeb91642dde57ca620b/users/a069c3de26ef47e9b581461d317dc28e/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:40:18.882367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245c8a2-8c9a-470a-aed5-8cde8ccfbc3d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:18.901749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7245c8a2-8c9a-470a-aed5-8cde8ccfbc3d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:18.976765 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d028cb-0126-4b26-8e54-ae0f7f65a1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:18.977436 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d028cb-0126-4b26-8e54-ae0f7f65a1c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:18.977720 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d028cb-0126-4b26-8e54-ae0f7f65a1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:18.977997 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d028cb-0126-4b26-8e54-ae0f7f65a1c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:18.978712 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d028cb-0126-4b26-8e54-ae0f7f65a1c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:18.980396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d028cb-0126-4b26-8e54-ae0f7f65a1c4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:18.991428 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1243/2452] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:40:18 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:40:18.995675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec0f6a-a54a-4fbe-8a3e-55f8afcbe457 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:19.027555 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec0f6a-a54a-4fbe-8a3e-55f8afcbe457 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:19.032987 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460835a-5f30-402a-899f-773576c1818f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:19.033805 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460835a-5f30-402a-899f-773576c1818f admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:19.034092 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460835a-5f30-402a-899f-773576c1818f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:19.034371 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1460835a-5f30-402a-899f-773576c1818f admin admin] PATH_INFO: `/v3/projects/a7b751318a744aeb91642dde57ca620b/users/a069c3de26ef47e9b581461d317dc28e/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:19.036651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460835a-5f30-402a-899f-773576c1818f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7b751318a744aeb91642dde57ca620b, user_id=a069c3de26ef47e9b581461d317dc28e, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:19.039409 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1460835a-5f30-402a-899f-773576c1818f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:19.053133 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1210/2453] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:40:18 2022] PUT /identity/v3/projects/a7b751318a744aeb91642dde57ca620b/users/a069c3de26ef47e9b581461d317dc28e/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:40:19.064977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bde2f-59a7-471b-ab02-0be3fc86f601 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:19.065324 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bde2f-59a7-471b-ab02-0be3fc86f601 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:19.065620 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bde2f-59a7-471b-ab02-0be3fc86f601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:19.095656 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-162bde2f-59a7-471b-ab02-0be3fc86f601 None None] MFA Rules not processed for user `a069c3de26ef47e9b581461d317dc28e`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:40:19.096915 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-162bde2f-59a7-471b-ab02-0be3fc86f601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:19.201067 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:40:19.217973 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1244/2454] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:40:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:40:19.225939 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460835a-5f30-402a-899f-773576c1818f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:19.227803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1460835a-5f30-402a-899f-773576c1818f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:19.314934 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1460835a-5f30-402a-899f-773576c1818f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:19.318718 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5ec29-1dd6-455f-a761-4542dfe00f83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:19.319377 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5ec29-1dd6-455f-a761-4542dfe00f83 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:19.319694 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5ec29-1dd6-455f-a761-4542dfe00f83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:19.320019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5ec29-1dd6-455f-a761-4542dfe00f83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:19.329993 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:40:19.383438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5ec29-1dd6-455f-a761-4542dfe00f83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:19.385719 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5ec29-1dd6-455f-a761-4542dfe00f83 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:19.396654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1211/2455] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:40:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:40:19.822381 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a1d8a-0bc1-4356-9e92-dd6ad0565a52 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:19.822381 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a1d8a-0bc1-4356-9e92-dd6ad0565a52 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:19.822381 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a1d8a-0bc1-4356-9e92-dd6ad0565a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:19.838334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-690a1d8a-0bc1-4356-9e92-dd6ad0565a52 None None] MFA Rules not processed for user `d49604bffa3d4e89a631eefa6d4e4f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:40:19.839604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-690a1d8a-0bc1-4356-9e92-dd6ad0565a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:19.898335 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1245/2456] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 30 16:40:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:40:20.058053 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5ec29-1dd6-455f-a761-4542dfe00f83 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:20.059256 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-cfe5ec29-1dd6-455f-a761-4542dfe00f83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:20.132911 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5ec29-1dd6-455f-a761-4542dfe00f83 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:20.138327 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb524435-74d2-45b1-8fd0-b008bd4a8e23 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988b917eea4b48ffae32e23b8555a5c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:20.138984 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb524435-74d2-45b1-8fd0-b008bd4a8e23 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:20.139266 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb524435-74d2-45b1-8fd0-b008bd4a8e23 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:20.139577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb524435-74d2-45b1-8fd0-b008bd4a8e23 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:20.151713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb524435-74d2-45b1-8fd0-b008bd4a8e23 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:20.153365 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb524435-74d2-45b1-8fd0-b008bd4a8e23 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:20.164567 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1212/2457] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 30 16:40:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 16:40:21.450968 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a1d8a-0bc1-4356-9e92-dd6ad0565a52 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:21.452654 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-690a1d8a-0bc1-4356-9e92-dd6ad0565a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:21.466090 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a1d8a-0bc1-4356-9e92-dd6ad0565a52 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:21.469981 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34612545-d343-4245-b8b4-6d9c9621154d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:21.470706 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34612545-d343-4245-b8b4-6d9c9621154d service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:21.471056 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34612545-d343-4245-b8b4-6d9c9621154d service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:21.472597 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34612545-d343-4245-b8b4-6d9c9621154d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:21.545095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34612545-d343-4245-b8b4-6d9c9621154d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:21.546872 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34612545-d343-4245-b8b4-6d9c9621154d service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:21.559316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1246/2458] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:40:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:40:23.989012 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb524435-74d2-45b1-8fd0-b008bd4a8e23 service swift] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:24.002053 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb524435-74d2-45b1-8fd0-b008bd4a8e23 service swift] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:24.011543 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d77333-9441-43fc-b966-15f7be165c07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:24.012303 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d77333-9441-43fc-b966-15f7be165c07 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:24.012649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d77333-9441-43fc-b966-15f7be165c07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:24.012996 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d77333-9441-43fc-b966-15f7be165c07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:24.024898 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d77333-9441-43fc-b966-15f7be165c07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:24.027573 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d77333-9441-43fc-b966-15f7be165c07 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:24.051172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1213/2459] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:40:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:40:30.193107 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34612545-d343-4245-b8b4-6d9c9621154d service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:30.260409 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34612545-d343-4245-b8b4-6d9c9621154d service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:30.264890 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107d9e7-2f1f-4c82-97c7-bb76dfd5ac46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:30.265753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107d9e7-2f1f-4c82-97c7-bb76dfd5ac46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:30.266032 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107d9e7-2f1f-4c82-97c7-bb76dfd5ac46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:30.266303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107d9e7-2f1f-4c82-97c7-bb76dfd5ac46 admin admin] PATH_INFO: `/v3/users/895345c685304bbba6088e98b3c3174d` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:30.267488 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107d9e7-2f1f-4c82-97c7-bb76dfd5ac46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=895345c685304bbba6088e98b3c3174d)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:30.269966 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107d9e7-2f1f-4c82-97c7-bb76dfd5ac46 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:30.502527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:40:30.513245 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3107d9e7-2f1f-4c82-97c7-bb76dfd5ac46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '895345c685304bbba6088e98b3c3174d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:30.514406 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3107d9e7-2f1f-4c82-97c7-bb76dfd5ac46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '895345c685304bbba6088e98b3c3174d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:30.553574 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3107d9e7-2f1f-4c82-97c7-bb76dfd5ac46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '895345c685304bbba6088e98b3c3174d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:30.619364 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3107d9e7-2f1f-4c82-97c7-bb76dfd5ac46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '895345c685304bbba6088e98b3c3174d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:30.620938 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3107d9e7-2f1f-4c82-97c7-bb76dfd5ac46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '895345c685304bbba6088e98b3c3174d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:30.656955 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1247/2460] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:40:30 2022] DELETE /identity/v3/users/895345c685304bbba6088e98b3c3174d => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:40:30.665475 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c401f-3d85-4c0b-8ba0-3867fea9f942 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:30.665951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c401f-3d85-4c0b-8ba0-3867fea9f942 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:30.666241 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891c401f-3d85-4c0b-8ba0-3867fea9f942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:30.701658 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-891c401f-3d85-4c0b-8ba0-3867fea9f942 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:40:30.707182 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-891c401f-3d85-4c0b-8ba0-3867fea9f942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:30.831864 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1214/2461] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:40:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:40:30.841338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107d9e7-2f1f-4c82-97c7-bb76dfd5ac46 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:30.843067 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3107d9e7-2f1f-4c82-97c7-bb76dfd5ac46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:30.917269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107d9e7-2f1f-4c82-97c7-bb76dfd5ac46 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:30.921598 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa58d9-b6ab-46ee-8f48-eee1798b7d43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:30.922267 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa58d9-b6ab-46ee-8f48-eee1798b7d43 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:30.922551 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa58d9-b6ab-46ee-8f48-eee1798b7d43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:30.922834 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aa58d9-b6ab-46ee-8f48-eee1798b7d43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:30.936375 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aa58d9-b6ab-46ee-8f48-eee1798b7d43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:30.938249 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aa58d9-b6ab-46ee-8f48-eee1798b7d43 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:30.951230 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1248/2462] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:40:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:40:31.512768 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c401f-3d85-4c0b-8ba0-3867fea9f942 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:31.512768 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-891c401f-3d85-4c0b-8ba0-3867fea9f942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:31.521483 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891c401f-3d85-4c0b-8ba0-3867fea9f942 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:31.526625 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34b822-f248-421b-a45e-9dc6bd970647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:31.527802 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34b822-f248-421b-a45e-9dc6bd970647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:31.528261 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34b822-f248-421b-a45e-9dc6bd970647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:31.528544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe34b822-f248-421b-a45e-9dc6bd970647 admin admin] PATH_INFO: `/v3/projects/9469bcd48a124cdf9c4c5e50ad1abcc9` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:31.529947 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe34b822-f248-421b-a45e-9dc6bd970647 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9469bcd48a124cdf9c4c5e50ad1abcc9)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:31.532317 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe34b822-f248-421b-a45e-9dc6bd970647 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:31.559079 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:40:31.702118 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe34b822-f248-421b-a45e-9dc6bd970647 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9469bcd48a124cdf9c4c5e50ad1abcc9', 'request_id': 'req-fe34b822-f248-421b-a45e-9dc6bd970647'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:31.709864 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe34b822-f248-421b-a45e-9dc6bd970647 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9469bcd48a124cdf9c4c5e50ad1abcc9', 'request_id': 'req-fe34b822-f248-421b-a45e-9dc6bd970647'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:31.717999 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe34b822-f248-421b-a45e-9dc6bd970647 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9469bcd48a124cdf9c4c5e50ad1abcc9', 'request_id': 'req-fe34b822-f248-421b-a45e-9dc6bd970647'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:31.808086 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe34b822-f248-421b-a45e-9dc6bd970647 admin admin] The token cache is being invalidate because project 9469bcd48a124cdf9c4c5e50ad1abcc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:40:31.808709 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe34b822-f248-421b-a45e-9dc6bd970647 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:31.809435 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe34b822-f248-421b-a45e-9dc6bd970647 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:31.811301 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1215/2463] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:40:31 2022] DELETE /identity/v3/projects/9469bcd48a124cdf9c4c5e50ad1abcc9 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:40:31.816455 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa58d9-b6ab-46ee-8f48-eee1798b7d43 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:31.828544 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aa58d9-b6ab-46ee-8f48-eee1798b7d43 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:31.902175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9a622-7fc7-4777-90ab-188686c248a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:31.903633 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9a622-7fc7-4777-90ab-188686c248a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:31.908147 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9a622-7fc7-4777-90ab-188686c248a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:31.908147 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc9a622-7fc7-4777-90ab-188686c248a8 admin admin] PATH_INFO: `/v3/users/a96a4ddbd69b446da0f73de51cb1e2a3` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:31.908147 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc9a622-7fc7-4777-90ab-188686c248a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a96a4ddbd69b446da0f73de51cb1e2a3)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:31.908147 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc9a622-7fc7-4777-90ab-188686c248a8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:32.177362 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cbc9a622-7fc7-4777-90ab-188686c248a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a96a4ddbd69b446da0f73de51cb1e2a3'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:32.178055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cbc9a622-7fc7-4777-90ab-188686c248a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a96a4ddbd69b446da0f73de51cb1e2a3'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:32.185120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cbc9a622-7fc7-4777-90ab-188686c248a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a96a4ddbd69b446da0f73de51cb1e2a3'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:32.275821 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cbc9a622-7fc7-4777-90ab-188686c248a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a96a4ddbd69b446da0f73de51cb1e2a3'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:32.283102 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cbc9a622-7fc7-4777-90ab-188686c248a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a96a4ddbd69b446da0f73de51cb1e2a3'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:32.283102 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:40:32.297090 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1249/2464] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:40:31 2022] DELETE /identity/v3/users/a96a4ddbd69b446da0f73de51cb1e2a3 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:40:32.964694 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34b822-f248-421b-a45e-9dc6bd970647 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:32.969676 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fe34b822-f248-421b-a45e-9dc6bd970647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:32.980119 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe34b822-f248-421b-a45e-9dc6bd970647 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:33.060751 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91a864-5520-489d-8b52-dc206507fe07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:33.061627 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91a864-5520-489d-8b52-dc206507fe07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:33.061907 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91a864-5520-489d-8b52-dc206507fe07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:33.062181 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f91a864-5520-489d-8b52-dc206507fe07 admin admin] PATH_INFO: `/v3/projects/7f70f01ea36744929f202f016184e71f` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:33.063800 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91a864-5520-489d-8b52-dc206507fe07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f70f01ea36744929f202f016184e71f)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:33.066681 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f91a864-5520-489d-8b52-dc206507fe07 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:33.226158 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8f91a864-5520-489d-8b52-dc206507fe07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f70f01ea36744929f202f016184e71f', 'request_id': 'req-8f91a864-5520-489d-8b52-dc206507fe07'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:33.232881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8f91a864-5520-489d-8b52-dc206507fe07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f70f01ea36744929f202f016184e71f', 'request_id': 'req-8f91a864-5520-489d-8b52-dc206507fe07'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:33.240650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8f91a864-5520-489d-8b52-dc206507fe07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f70f01ea36744929f202f016184e71f', 'request_id': 'req-8f91a864-5520-489d-8b52-dc206507fe07'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:33.317854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8f91a864-5520-489d-8b52-dc206507fe07 admin admin] The token cache is being invalidate because project 7f70f01ea36744929f202f016184e71f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:40:33.317854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8f91a864-5520-489d-8b52-dc206507fe07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:33.318240 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8f91a864-5520-489d-8b52-dc206507fe07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:33.319531 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1216/2465] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:40:32 2022] DELETE /identity/v3/projects/7f70f01ea36744929f202f016184e71f => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:40:33.323904 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9a622-7fc7-4777-90ab-188686c248a8 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:33.325385 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cbc9a622-7fc7-4777-90ab-188686c248a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:33.339192 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc9a622-7fc7-4777-90ab-188686c248a8 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:33.411119 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4e41f-3709-4139-94e5-b5bcb78cd9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:33.412162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4e41f-3709-4139-94e5-b5bcb78cd9f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:33.412442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4e41f-3709-4139-94e5-b5bcb78cd9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:33.412718 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4e41f-3709-4139-94e5-b5bcb78cd9f1 admin admin] PATH_INFO: `/v3/users/a069c3de26ef47e9b581461d317dc28e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:33.414043 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4e41f-3709-4139-94e5-b5bcb78cd9f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a069c3de26ef47e9b581461d317dc28e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:33.416960 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4e41f-3709-4139-94e5-b5bcb78cd9f1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:33.644909 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a1d4e41f-3709-4139-94e5-b5bcb78cd9f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a069c3de26ef47e9b581461d317dc28e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:33.645510 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a1d4e41f-3709-4139-94e5-b5bcb78cd9f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a069c3de26ef47e9b581461d317dc28e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:33.652720 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a1d4e41f-3709-4139-94e5-b5bcb78cd9f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a069c3de26ef47e9b581461d317dc28e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:33.722310 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a1d4e41f-3709-4139-94e5-b5bcb78cd9f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a069c3de26ef47e9b581461d317dc28e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:33.722976 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a1d4e41f-3709-4139-94e5-b5bcb78cd9f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a069c3de26ef47e9b581461d317dc28e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:33.743022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1250/2466] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:40:33 2022] DELETE /identity/v3/users/a069c3de26ef47e9b581461d317dc28e => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:40:34.343616 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91a864-5520-489d-8b52-dc206507fe07 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:34.345144 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8f91a864-5520-489d-8b52-dc206507fe07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:34.360649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f91a864-5520-489d-8b52-dc206507fe07 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:34.421313 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:40:34.444901 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf0dad-d57e-4c32-81d6-7701f05716d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:34.445788 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf0dad-d57e-4c32-81d6-7701f05716d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:34.446077 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf0dad-d57e-4c32-81d6-7701f05716d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:34.446356 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf0dad-d57e-4c32-81d6-7701f05716d9 admin admin] PATH_INFO: `/v3/projects/a7b751318a744aeb91642dde57ca620b` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:34.447780 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf0dad-d57e-4c32-81d6-7701f05716d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7b751318a744aeb91642dde57ca620b)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:34.449712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf0dad-d57e-4c32-81d6-7701f05716d9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:34.602613 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3bdf0dad-d57e-4c32-81d6-7701f05716d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7b751318a744aeb91642dde57ca620b', 'request_id': 'req-3bdf0dad-d57e-4c32-81d6-7701f05716d9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:34.609363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3bdf0dad-d57e-4c32-81d6-7701f05716d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7b751318a744aeb91642dde57ca620b', 'request_id': 'req-3bdf0dad-d57e-4c32-81d6-7701f05716d9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:34.619196 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3bdf0dad-d57e-4c32-81d6-7701f05716d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7b751318a744aeb91642dde57ca620b', 'request_id': 'req-3bdf0dad-d57e-4c32-81d6-7701f05716d9'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:34.722051 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3bdf0dad-d57e-4c32-81d6-7701f05716d9 admin admin] The token cache is being invalidate because project a7b751318a744aeb91642dde57ca620b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:40:34.722336 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3bdf0dad-d57e-4c32-81d6-7701f05716d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:34.723025 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3bdf0dad-d57e-4c32-81d6-7701f05716d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:34.724617 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1217/2467] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:40:34 2022] DELETE /identity/v3/projects/a7b751318a744aeb91642dde57ca620b => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:40:34.729547 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab779222-2837-4398-8896-fa1750d51733 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:34.730010 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab779222-2837-4398-8896-fa1750d51733 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:34.730180 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab779222-2837-4398-8896-fa1750d51733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:34.751121 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ab779222-2837-4398-8896-fa1750d51733 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:40:34.752507 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ab779222-2837-4398-8896-fa1750d51733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:34.818369 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:40:34.827176 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1251/2468] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:40:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:40:34.836811 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb748b-7097-4061-a37d-05f41b8a6b4e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:34.836811 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb748b-7097-4061-a37d-05f41b8a6b4e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:34.837161 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb748b-7097-4061-a37d-05f41b8a6b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:34.858476 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c1cb748b-7097-4061-a37d-05f41b8a6b4e None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:40:34.863516 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c1cb748b-7097-4061-a37d-05f41b8a6b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:34.869422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1218/2469] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:40:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:40:34.873753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab779222-2837-4398-8896-fa1750d51733 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:34.884803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab779222-2837-4398-8896-fa1750d51733 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:34.895075 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49c89d-6287-465c-9cb2-1c87aa124f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:34.895075 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49c89d-6287-465c-9cb2-1c87aa124f06 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:34.895075 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49c89d-6287-465c-9cb2-1c87aa124f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:34.895075 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49c89d-6287-465c-9cb2-1c87aa124f06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:34.895457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49c89d-6287-465c-9cb2-1c87aa124f06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:40:34.895457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49c89d-6287-465c-9cb2-1c87aa124f06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:34.895457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c49c89d-6287-465c-9cb2-1c87aa124f06 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:34.909602 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1252/2470] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:40:34.914072 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb748b-7097-4061-a37d-05f41b8a6b4e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:34.923862 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb748b-7097-4061-a37d-05f41b8a6b4e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:34.929359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5eeb4-1a10-497a-85ad-51ce1c7b12ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:34.930073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5eeb4-1a10-497a-85ad-51ce1c7b12ff admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:34.930372 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5eeb4-1a10-497a-85ad-51ce1c7b12ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:34.930647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac5eeb4-1a10-497a-85ad-51ce1c7b12ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:34.931360 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac5eeb4-1a10-497a-85ad-51ce1c7b12ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:34.933502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac5eeb4-1a10-497a-85ad-51ce1c7b12ff admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:35.118787 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1219/2471] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:40:34 2022] POST /identity/v3/projects => generated 373 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:40:35.122959 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49c89d-6287-465c-9cb2-1c87aa124f06 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:35.134120 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c49c89d-6287-465c-9cb2-1c87aa124f06 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:35.193745 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbda28-6830-4fcd-8e78-efbfea191141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:35.194422 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbda28-6830-4fcd-8e78-efbfea191141 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:35.194877 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbda28-6830-4fcd-8e78-efbfea191141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:35.195154 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbda28-6830-4fcd-8e78-efbfea191141 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:35.195931 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbda28-6830-4fcd-8e78-efbfea191141 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:35.197886 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbda28-6830-4fcd-8e78-efbfea191141 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:35.283732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1253/2472] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:40:35 2022] POST /identity/v3/users => generated 287 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:40:35.285586 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5eeb4-1a10-497a-85ad-51ce1c7b12ff admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:35.296776 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac5eeb4-1a10-497a-85ad-51ce1c7b12ff admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:35.302544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88700a36-02b7-4341-9de6-cc75834c1d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:35.303220 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88700a36-02b7-4341-9de6-cc75834c1d32 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:35.303535 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88700a36-02b7-4341-9de6-cc75834c1d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:35.307100 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88700a36-02b7-4341-9de6-cc75834c1d32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:35.307100 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88700a36-02b7-4341-9de6-cc75834c1d32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:35.307100 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88700a36-02b7-4341-9de6-cc75834c1d32 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:35.320294 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1220/2473] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:40:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:40:35.324961 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbda28-6830-4fcd-8e78-efbfea191141 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:35.336307 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbda28-6830-4fcd-8e78-efbfea191141 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:35.344220 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cae887-66ee-4e61-a312-4a40dd184dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:35.344220 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cae887-66ee-4e61-a312-4a40dd184dda admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:35.344755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cae887-66ee-4e61-a312-4a40dd184dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:35.355981 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cae887-66ee-4e61-a312-4a40dd184dda admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:35.356775 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cae887-66ee-4e61-a312-4a40dd184dda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:35.359656 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cae887-66ee-4e61-a312-4a40dd184dda admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:35.372427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1254/2474] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:40:35 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:40:35.377006 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88700a36-02b7-4341-9de6-cc75834c1d32 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:35.391572 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88700a36-02b7-4341-9de6-cc75834c1d32 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:35.397525 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b35f2e-c9a4-495e-9e05-2b5e6f70fc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:35.398480 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b35f2e-c9a4-495e-9e05-2b5e6f70fc76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:35.398770 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b35f2e-c9a4-495e-9e05-2b5e6f70fc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:35.399047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b35f2e-c9a4-495e-9e05-2b5e6f70fc76 admin admin] PATH_INFO: `/v3/projects/82518f19d8674949bee4125558da0c24/users/da8bc7ff37e1473494183f058b1c57a7/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:35.420082 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b35f2e-c9a4-495e-9e05-2b5e6f70fc76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82518f19d8674949bee4125558da0c24, user_id=da8bc7ff37e1473494183f058b1c57a7, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:35.422797 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b35f2e-c9a4-495e-9e05-2b5e6f70fc76 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:35.517900 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1221/2475] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:40:35 2022] PUT /identity/v3/projects/82518f19d8674949bee4125558da0c24/users/da8bc7ff37e1473494183f058b1c57a7/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:40:35.528789 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b3e54-7155-4a67-913b-3ca24b37d6a2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:35.529103 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b3e54-7155-4a67-913b-3ca24b37d6a2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:35.529386 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b3e54-7155-4a67-913b-3ca24b37d6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:35.557804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-136b3e54-7155-4a67-913b-3ca24b37d6a2 None None] MFA Rules not processed for user `da8bc7ff37e1473494183f058b1c57a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:40:35.559094 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-136b3e54-7155-4a67-913b-3ca24b37d6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:35.630589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1255/2476] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:40:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:40:35.641327 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b35f2e-c9a4-495e-9e05-2b5e6f70fc76 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:35.643043 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-74b35f2e-c9a4-495e-9e05-2b5e6f70fc76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:35.706327 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b35f2e-c9a4-495e-9e05-2b5e6f70fc76 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:35.710364 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd4510-df7c-40c4-b8f3-8ef12ff59b99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:35.711019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd4510-df7c-40c4-b8f3-8ef12ff59b99 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:35.711302 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd4510-df7c-40c4-b8f3-8ef12ff59b99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:35.711605 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd4510-df7c-40c4-b8f3-8ef12ff59b99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:35.720781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd4510-df7c-40c4-b8f3-8ef12ff59b99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:35.723265 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd4510-df7c-40c4-b8f3-8ef12ff59b99 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:35.733355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1222/2477] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:40:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:40:37.976748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69219cb-1449-457c-9254-c3c69044226b None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:37.977294 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69219cb-1449-457c-9254-c3c69044226b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:37.977405 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69219cb-1449-457c-9254-c3c69044226b None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:37.978753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1256/2478] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 30 16:40:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:40:37.985387 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08da02b-80f1-4346-8d26-edb6cc584107 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:37.985870 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08da02b-80f1-4346-8d26-edb6cc584107 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:37.985969 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08da02b-80f1-4346-8d26-edb6cc584107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:38.005993 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d08da02b-80f1-4346-8d26-edb6cc584107 None None] MFA Rules not processed for user `71cca3708386459197abf6c67505bc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:40:38.008046 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d08da02b-80f1-4346-8d26-edb6cc584107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:38.013162 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:40:38.077513 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1223/2479] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 30 16:40:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:40:38.090242 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69219cb-1449-457c-9254-c3c69044226b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:38.092146 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d69219cb-1449-457c-9254-c3c69044226b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:38.153998 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69219cb-1449-457c-9254-c3c69044226b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:38.157539 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1fc785-7f6e-4245-bce1-bb891988afce service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec41c84c9375437590c42621ca5b400a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:38.158122 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1fc785-7f6e-4245-bce1-bb891988afce service glance] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:38.158401 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1fc785-7f6e-4245-bce1-bb891988afce service glance] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:38.158681 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1fc785-7f6e-4245-bce1-bb891988afce service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:38.170126 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1fc785-7f6e-4245-bce1-bb891988afce service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:38.171587 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1fc785-7f6e-4245-bce1-bb891988afce service glance] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:38.177463 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:40:38.183635 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1257/2480] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:40:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:40:38.447390 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b500ac-a9ce-4aec-8b58-a4dd985aca94 None None] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:38.447390 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b500ac-a9ce-4aec-8b58-a4dd985aca94 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:38.447390 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b500ac-a9ce-4aec-8b58-a4dd985aca94 None None] PATH_INFO: `/` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:38.447390 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1224/2481] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 30 16:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:40:38.452183 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7a71e-cf25-4405-81a6-89dfd7363c4d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:38.452552 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7a71e-cf25-4405-81a6-89dfd7363c4d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:38.452944 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7a71e-cf25-4405-81a6-89dfd7363c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:38.472755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1ab7a71e-cf25-4405-81a6-89dfd7363c4d None None] MFA Rules not processed for user `9b508dd8b7244d5f979e1ce7787d2c67`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:40:38.473983 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1ab7a71e-cf25-4405-81a6-89dfd7363c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:38.528838 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1258/2482] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 30 16:40:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:40:38.542170 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b500ac-a9ce-4aec-8b58-a4dd985aca94 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:38.543964 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e7b500ac-a9ce-4aec-8b58-a4dd985aca94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:38.557904 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b500ac-a9ce-4aec-8b58-a4dd985aca94 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:38.562519 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae45d3-c9cf-445a-ae7d-e0e60ec22078 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:38.562519 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae45d3-c9cf-445a-ae7d-e0e60ec22078 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:38.562519 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae45d3-c9cf-445a-ae7d-e0e60ec22078 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:38.563175 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae45d3-c9cf-445a-ae7d-e0e60ec22078 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:38.571251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae45d3-c9cf-445a-ae7d-e0e60ec22078 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:38.572850 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae45d3-c9cf-445a-ae7d-e0e60ec22078 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:38.583219 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1225/2483] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:40:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:40:38.703174 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d96b0-4cfe-47f8-8653-82c3b0e5d3d3 None None] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:38.703492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d96b0-4cfe-47f8-8653-82c3b0e5d3d3 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:38.703774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16d96b0-4cfe-47f8-8653-82c3b0e5d3d3 None None] PATH_INFO: `/` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:38.705141 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1259/2484] 10.222.0.67 () {54 vars in 921 bytes} [Thu Jun 30 16:40:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 16:40:38.716549 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6a671-601f-42e2-9716-96672274016c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:38.717045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6a671-601f-42e2-9716-96672274016c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:38.717600 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f6a671-601f-42e2-9716-96672274016c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:38.742203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-60f6a671-601f-42e2-9716-96672274016c None None] MFA Rules not processed for user `71cca3708386459197abf6c67505bc5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:40:38.743408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-60f6a671-601f-42e2-9716-96672274016c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:38.752682 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1226/2485] 10.222.0.67 () {58 vars in 1018 bytes} [Thu Jun 30 16:40:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:40:38.764216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d96b0-4cfe-47f8-8653-82c3b0e5d3d3 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:38.775954 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16d96b0-4cfe-47f8-8653-82c3b0e5d3d3 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:38.779610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3924cb-4aa0-415e-a149-aa2ebd6d4c37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:38.780400 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3924cb-4aa0-415e-a149-aa2ebd6d4c37 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:38.780685 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3924cb-4aa0-415e-a149-aa2ebd6d4c37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:38.780960 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3924cb-4aa0-415e-a149-aa2ebd6d4c37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:38.791039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3924cb-4aa0-415e-a149-aa2ebd6d4c37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:38.792505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3924cb-4aa0-415e-a149-aa2ebd6d4c37 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:38.803040 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1260/2486] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:40:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:40:41.763130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece46fb-2e04-4443-8bfb-ce63b1b6054f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:41.767741 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece46fb-2e04-4443-8bfb-ce63b1b6054f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:41.767741 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece46fb-2e04-4443-8bfb-ce63b1b6054f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:41.783753 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0ece46fb-2e04-4443-8bfb-ce63b1b6054f None None] MFA Rules not processed for user `d49604bffa3d4e89a631eefa6d4e4f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:40:41.785828 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0ece46fb-2e04-4443-8bfb-ce63b1b6054f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:41.867048 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1227/2487] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 30 16:40:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:40:41.890148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3924cb-4aa0-415e-a149-aa2ebd6d4c37 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:41.891881 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3a3924cb-4aa0-415e-a149-aa2ebd6d4c37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:41.978744 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3924cb-4aa0-415e-a149-aa2ebd6d4c37 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:41.983760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6b721-0401-49ab-abd3-0a35cc7e773d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988b917eea4b48ffae32e23b8555a5c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:41.984405 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6b721-0401-49ab-abd3-0a35cc7e773d service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:41.984689 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6b721-0401-49ab-abd3-0a35cc7e773d service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:41.984967 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6b721-0401-49ab-abd3-0a35cc7e773d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:42.005410 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f6b721-0401-49ab-abd3-0a35cc7e773d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:42.007084 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f6b721-0401-49ab-abd3-0a35cc7e773d service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:42.020143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1261/2488] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 30 16:40:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 16:40:48.619099 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece46fb-2e04-4443-8bfb-ce63b1b6054f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:48.706004 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece46fb-2e04-4443-8bfb-ce63b1b6054f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:48.711957 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2dc457-f1ec-4bb3-9268-6103fef84894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:48.712863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dc457-f1ec-4bb3-9268-6103fef84894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:48.713149 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dc457-f1ec-4bb3-9268-6103fef84894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:48.713429 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2dc457-f1ec-4bb3-9268-6103fef84894 admin admin] PATH_INFO: `/v3/users/da8bc7ff37e1473494183f058b1c57a7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:48.714658 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2dc457-f1ec-4bb3-9268-6103fef84894 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da8bc7ff37e1473494183f058b1c57a7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:48.716720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2dc457-f1ec-4bb3-9268-6103fef84894 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:49.358005 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:40:49.374230 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7c2dc457-f1ec-4bb3-9268-6103fef84894 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da8bc7ff37e1473494183f058b1c57a7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:49.374998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7c2dc457-f1ec-4bb3-9268-6103fef84894 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da8bc7ff37e1473494183f058b1c57a7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:49.437562 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7c2dc457-f1ec-4bb3-9268-6103fef84894 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da8bc7ff37e1473494183f058b1c57a7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:49.447261 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7c2dc457-f1ec-4bb3-9268-6103fef84894 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da8bc7ff37e1473494183f058b1c57a7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:49.448052 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7c2dc457-f1ec-4bb3-9268-6103fef84894 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da8bc7ff37e1473494183f058b1c57a7'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:49.482548 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1228/2489] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:40:48 2022] DELETE /identity/v3/users/da8bc7ff37e1473494183f058b1c57a7 => generated 0 bytes in 864 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:40:49.487014 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90faa8b4-fca3-40e9-a01b-ae59e38e1430 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:49.488325 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90faa8b4-fca3-40e9-a01b-ae59e38e1430 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:49.488325 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90faa8b4-fca3-40e9-a01b-ae59e38e1430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:49.509702 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-90faa8b4-fca3-40e9-a01b-ae59e38e1430 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:40:49.511880 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-90faa8b4-fca3-40e9-a01b-ae59e38e1430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:49.605397 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1262/2490] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:40:49.616398 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2dc457-f1ec-4bb3-9268-6103fef84894 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:49.617966 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7c2dc457-f1ec-4bb3-9268-6103fef84894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:49.681715 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2dc457-f1ec-4bb3-9268-6103fef84894 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:49.685693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6a839f-8166-4510-ac5a-34b71a0a2ae6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:49.686422 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a839f-8166-4510-ac5a-34b71a0a2ae6 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:49.686775 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a839f-8166-4510-ac5a-34b71a0a2ae6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:49.687238 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6a839f-8166-4510-ac5a-34b71a0a2ae6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:49.698753 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a839f-8166-4510-ac5a-34b71a0a2ae6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:49.700550 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6a839f-8166-4510-ac5a-34b71a0a2ae6 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:49.715638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1229/2491] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:40:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:40:50.211114 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90faa8b4-fca3-40e9-a01b-ae59e38e1430 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:50.212579 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-90faa8b4-fca3-40e9-a01b-ae59e38e1430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:50.226530 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90faa8b4-fca3-40e9-a01b-ae59e38e1430 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:50.232497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda13b58-cb15-4413-bc52-456f26b0f7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:50.233535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda13b58-cb15-4413-bc52-456f26b0f7e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:50.233820 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda13b58-cb15-4413-bc52-456f26b0f7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:50.234091 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda13b58-cb15-4413-bc52-456f26b0f7e5 admin admin] PATH_INFO: `/v3/projects/82518f19d8674949bee4125558da0c24` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:50.235357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda13b58-cb15-4413-bc52-456f26b0f7e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82518f19d8674949bee4125558da0c24)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:50.237344 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda13b58-cb15-4413-bc52-456f26b0f7e5 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:50.267839 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:40:50.359731 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bda13b58-cb15-4413-bc52-456f26b0f7e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82518f19d8674949bee4125558da0c24', 'request_id': 'req-bda13b58-cb15-4413-bc52-456f26b0f7e5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:50.443081 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bda13b58-cb15-4413-bc52-456f26b0f7e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82518f19d8674949bee4125558da0c24', 'request_id': 'req-bda13b58-cb15-4413-bc52-456f26b0f7e5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:50.450855 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bda13b58-cb15-4413-bc52-456f26b0f7e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82518f19d8674949bee4125558da0c24', 'request_id': 'req-bda13b58-cb15-4413-bc52-456f26b0f7e5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:50.462003 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bda13b58-cb15-4413-bc52-456f26b0f7e5 admin admin] The token cache is being invalidate because project 82518f19d8674949bee4125558da0c24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:40:50.462270 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bda13b58-cb15-4413-bc52-456f26b0f7e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:50.462926 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-bda13b58-cb15-4413-bc52-456f26b0f7e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:40:50.464530 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1263/2492] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:40:50 2022] DELETE /identity/v3/projects/82518f19d8674949bee4125558da0c24 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:40:57.210452 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d09ef-c33b-49d5-9bd7-921b3aba7c1d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:57.211503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d09ef-c33b-49d5-9bd7-921b3aba7c1d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:57.211908 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d09ef-c33b-49d5-9bd7-921b3aba7c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:57.234228 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2b6d09ef-c33b-49d5-9bd7-921b3aba7c1d None None] MFA Rules not processed for user `d49604bffa3d4e89a631eefa6d4e4f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:40:57.235461 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2b6d09ef-c33b-49d5-9bd7-921b3aba7c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:57.298333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1230/2493] 10.222.0.67 () {58 vars in 967 bytes} [Thu Jun 30 16:40:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:40:57.312193 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda13b58-cb15-4413-bc52-456f26b0f7e5 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:40:57.314198 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bda13b58-cb15-4413-bc52-456f26b0f7e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:57.401791 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda13b58-cb15-4413-bc52-456f26b0f7e5 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:40:57.410849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35128a55-5cf4-4a33-bc4c-2497534b024c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988b917eea4b48ffae32e23b8555a5c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:40:57.411653 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35128a55-5cf4-4a33-bc4c-2497534b024c service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:57.411924 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35128a55-5cf4-4a33-bc4c-2497534b024c service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:57.412338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35128a55-5cf4-4a33-bc4c-2497534b024c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:57.424033 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35128a55-5cf4-4a33-bc4c-2497534b024c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:40:57.434679 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35128a55-5cf4-4a33-bc4c-2497534b024c service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:40:57.445846 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1264/2494] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 30 16:40:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 16:40:58.951293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce093f36-cbe7-48c9-aa8a-df8c90ea376e None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:40:58.951845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce093f36-cbe7-48c9-aa8a-df8c90ea376e None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:40:58.951961 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce093f36-cbe7-48c9-aa8a-df8c90ea376e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:40:58.969840 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ce093f36-cbe7-48c9-aa8a-df8c90ea376e None None] MFA Rules not processed for user `bb8a8a4e8b634322a92992379c98b721`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:40:58.971029 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ce093f36-cbe7-48c9-aa8a-df8c90ea376e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:40:59.028861 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:40:59.038102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1231/2495] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:40:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:07.436802 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:19.451793 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35128a55-5cf4-4a33-bc4c-2497534b024c service swift] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:19.453426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-35128a55-5cf4-4a33-bc4c-2497534b024c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:19.540204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35128a55-5cf4-4a33-bc4c-2497534b024c service swift] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:19.544969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354593c-9ee1-44c9-97a6-aeb32854c270 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:19.545964 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354593c-9ee1-44c9-97a6-aeb32854c270 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:19.546475 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354593c-9ee1-44c9-97a6-aeb32854c270 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:19.546973 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2354593c-9ee1-44c9-97a6-aeb32854c270 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:19.549169 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2354593c-9ee1-44c9-97a6-aeb32854c270 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:19.637243 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2354593c-9ee1-44c9-97a6-aeb32854c270 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:19.638626 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2354593c-9ee1-44c9-97a6-aeb32854c270 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:19.649640 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1265/2496] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:41:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:41:23.838018 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce093f36-cbe7-48c9-aa8a-df8c90ea376e None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:23.840177 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ce093f36-cbe7-48c9-aa8a-df8c90ea376e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:23.919648 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce093f36-cbe7-48c9-aa8a-df8c90ea376e None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:23.924814 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc75229-fc62-4ef1-9c37-57d3db493047 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a932a9390fdd47f3adcf053a60250944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:23.926071 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc75229-fc62-4ef1-9c37-57d3db493047 service placement] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:23.926529 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc75229-fc62-4ef1-9c37-57d3db493047 service placement] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:23.926781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc75229-fc62-4ef1-9c37-57d3db493047 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:23.928703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dfc75229-fc62-4ef1-9c37-57d3db493047 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:23.941972 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc75229-fc62-4ef1-9c37-57d3db493047 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:23.945590 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc75229-fc62-4ef1-9c37-57d3db493047 service placement] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:23.954823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1232/2497] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:41:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:41:24.267060 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354593c-9ee1-44c9-97a6-aeb32854c270 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:24.270856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2354593c-9ee1-44c9-97a6-aeb32854c270 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:24.287600 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2354593c-9ee1-44c9-97a6-aeb32854c270 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:24.292264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3864847-601b-4ad4-9be1-4d01ffba345a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:24.293145 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3864847-601b-4ad4-9be1-4d01ffba345a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:24.293423 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3864847-601b-4ad4-9be1-4d01ffba345a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:24.293702 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3864847-601b-4ad4-9be1-4d01ffba345a admin admin] PATH_INFO: `/v3/users/0a26028451b345968fe86774f44736dd` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:24.295036 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3864847-601b-4ad4-9be1-4d01ffba345a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a26028451b345968fe86774f44736dd)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:24.297009 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3864847-601b-4ad4-9be1-4d01ffba345a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:24.583183 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b3864847-601b-4ad4-9be1-4d01ffba345a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a26028451b345968fe86774f44736dd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:24.584429 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b3864847-601b-4ad4-9be1-4d01ffba345a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a26028451b345968fe86774f44736dd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:24.593488 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b3864847-601b-4ad4-9be1-4d01ffba345a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a26028451b345968fe86774f44736dd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:24.667121 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b3864847-601b-4ad4-9be1-4d01ffba345a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a26028451b345968fe86774f44736dd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:24.668202 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b3864847-601b-4ad4-9be1-4d01ffba345a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a26028451b345968fe86774f44736dd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:24.697065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1266/2498] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:41:24 2022] DELETE /identity/v3/users/0a26028451b345968fe86774f44736dd => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:25.535968 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc75229-fc62-4ef1-9c37-57d3db493047 service placement] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:25.549612 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc75229-fc62-4ef1-9c37-57d3db493047 service placement] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:25.635419 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115e3c5-d3ea-4a6c-8441-55a020012b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:25.636323 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115e3c5-d3ea-4a6c-8441-55a020012b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:25.636605 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115e3c5-d3ea-4a6c-8441-55a020012b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:25.636883 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115e3c5-d3ea-4a6c-8441-55a020012b91 admin admin] PATH_INFO: `/v3/projects/f942ab31bafc428bb1fba94aee2e0c32` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:25.638261 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115e3c5-d3ea-4a6c-8441-55a020012b91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f942ab31bafc428bb1fba94aee2e0c32)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:25.640199 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115e3c5-d3ea-4a6c-8441-55a020012b91 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:25.771986 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0115e3c5-d3ea-4a6c-8441-55a020012b91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f942ab31bafc428bb1fba94aee2e0c32', 'request_id': 'req-0115e3c5-d3ea-4a6c-8441-55a020012b91'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:25.778353 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0115e3c5-d3ea-4a6c-8441-55a020012b91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f942ab31bafc428bb1fba94aee2e0c32', 'request_id': 'req-0115e3c5-d3ea-4a6c-8441-55a020012b91'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:25.787423 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0115e3c5-d3ea-4a6c-8441-55a020012b91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f942ab31bafc428bb1fba94aee2e0c32', 'request_id': 'req-0115e3c5-d3ea-4a6c-8441-55a020012b91'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:25.889195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0115e3c5-d3ea-4a6c-8441-55a020012b91 admin admin] The token cache is being invalidate because project f942ab31bafc428bb1fba94aee2e0c32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:41:25.889494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0115e3c5-d3ea-4a6c-8441-55a020012b91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:25.893350 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-0115e3c5-d3ea-4a6c-8441-55a020012b91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:25.895068 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1233/2499] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:41:25 2022] DELETE /identity/v3/projects/f942ab31bafc428bb1fba94aee2e0c32 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:25.900219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6906b0-5cfe-4c3e-a7cc-8e2a50eadae2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:25.900840 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6906b0-5cfe-4c3e-a7cc-8e2a50eadae2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:25.901183 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6906b0-5cfe-4c3e-a7cc-8e2a50eadae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:25.921283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ea6906b0-5cfe-4c3e-a7cc-8e2a50eadae2 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:25.922521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ea6906b0-5cfe-4c3e-a7cc-8e2a50eadae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:25.943288 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:25.987188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1267/2500] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:25.996191 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991ec8b-26ef-4ed9-8080-5ddd2aafffca None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:25.996499 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991ec8b-26ef-4ed9-8080-5ddd2aafffca None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:25.996743 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991ec8b-26ef-4ed9-8080-5ddd2aafffca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:26.016720 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b991ec8b-26ef-4ed9-8080-5ddd2aafffca None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:26.017919 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b991ec8b-26ef-4ed9-8080-5ddd2aafffca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:26.033292 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1234/2501] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:26.037221 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6906b0-5cfe-4c3e-a7cc-8e2a50eadae2 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:26.047093 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6906b0-5cfe-4c3e-a7cc-8e2a50eadae2 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:26.052793 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2efe30-9755-4cc7-b61d-6ce7c3f13243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:26.053458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2efe30-9755-4cc7-b61d-6ce7c3f13243 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:26.053727 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2efe30-9755-4cc7-b61d-6ce7c3f13243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:26.054001 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2efe30-9755-4cc7-b61d-6ce7c3f13243 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:26.054709 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2efe30-9755-4cc7-b61d-6ce7c3f13243 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:41:26.055013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2efe30-9755-4cc7-b61d-6ce7c3f13243 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:26.056720 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2efe30-9755-4cc7-b61d-6ce7c3f13243 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:26.069527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1268/2502] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:41:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:41:26.073938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991ec8b-26ef-4ed9-8080-5ddd2aafffca None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:26.083879 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991ec8b-26ef-4ed9-8080-5ddd2aafffca None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:26.088534 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9da8f-97c9-4fa2-bd1a-fefcbde5a90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:26.089191 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9da8f-97c9-4fa2-bd1a-fefcbde5a90b admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:26.089469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9da8f-97c9-4fa2-bd1a-fefcbde5a90b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:26.089740 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9da8f-97c9-4fa2-bd1a-fefcbde5a90b admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:26.090444 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9da8f-97c9-4fa2-bd1a-fefcbde5a90b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:26.092367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9da8f-97c9-4fa2-bd1a-fefcbde5a90b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:26.103544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:26.171382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1235/2503] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:41:26 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:41:26.176542 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2efe30-9755-4cc7-b61d-6ce7c3f13243 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:26.187418 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2efe30-9755-4cc7-b61d-6ce7c3f13243 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:26.253498 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08079db5-568b-4552-9780-8973ed4e8fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:26.254430 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08079db5-568b-4552-9780-8973ed4e8fa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:26.254561 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08079db5-568b-4552-9780-8973ed4e8fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:26.254813 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08079db5-568b-4552-9780-8973ed4e8fa0 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:26.255547 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08079db5-568b-4552-9780-8973ed4e8fa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:26.257500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08079db5-568b-4552-9780-8973ed4e8fa0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:26.387635 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1269/2504] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:41:26 2022] POST /identity/v3/users => generated 287 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:41:26.392063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9da8f-97c9-4fa2-bd1a-fefcbde5a90b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:26.403542 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9da8f-97c9-4fa2-bd1a-fefcbde5a90b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:26.408906 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb249905-86d6-422e-b0c9-51c548115340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:26.409617 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb249905-86d6-422e-b0c9-51c548115340 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:26.410694 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb249905-86d6-422e-b0c9-51c548115340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:26.411170 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb249905-86d6-422e-b0c9-51c548115340 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:26.411948 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb249905-86d6-422e-b0c9-51c548115340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:26.413646 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb249905-86d6-422e-b0c9-51c548115340 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:26.425520 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1236/2505] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:41:26.430343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08079db5-568b-4552-9780-8973ed4e8fa0 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:26.441416 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08079db5-568b-4552-9780-8973ed4e8fa0 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:26.446496 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de617aca-0e5e-4d0c-84a8-e74a1007f361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:26.447205 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de617aca-0e5e-4d0c-84a8-e74a1007f361 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:26.447551 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de617aca-0e5e-4d0c-84a8-e74a1007f361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:26.447830 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de617aca-0e5e-4d0c-84a8-e74a1007f361 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:26.448567 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de617aca-0e5e-4d0c-84a8-e74a1007f361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:26.450191 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de617aca-0e5e-4d0c-84a8-e74a1007f361 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:26.462116 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1270/2506] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:41:26.466362 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb249905-86d6-422e-b0c9-51c548115340 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:26.477533 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb249905-86d6-422e-b0c9-51c548115340 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:26.482309 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b4e15-f3a2-496c-8b8b-306c68245126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:26.483224 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b4e15-f3a2-496c-8b8b-306c68245126 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:26.483538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b4e15-f3a2-496c-8b8b-306c68245126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:26.483814 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b4e15-f3a2-496c-8b8b-306c68245126 admin admin] PATH_INFO: `/v3/projects/561fc307f26d40b1a73b287ff89293f1/users/65fcedd910c6461d8d9a43efdef6154e/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:26.499819 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b4e15-f3a2-496c-8b8b-306c68245126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=561fc307f26d40b1a73b287ff89293f1, user_id=65fcedd910c6461d8d9a43efdef6154e, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:26.503129 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b4e15-f3a2-496c-8b8b-306c68245126 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:26.598205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1237/2507] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:41:26 2022] PUT /identity/v3/projects/561fc307f26d40b1a73b287ff89293f1/users/65fcedd910c6461d8d9a43efdef6154e/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:26.609625 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb97538-2752-4743-b77a-72b4e68e7067 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:26.609917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb97538-2752-4743-b77a-72b4e68e7067 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:26.610207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb97538-2752-4743-b77a-72b4e68e7067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:26.637492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5fb97538-2752-4743-b77a-72b4e68e7067 None None] MFA Rules not processed for user `65fcedd910c6461d8d9a43efdef6154e`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:26.638725 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5fb97538-2752-4743-b77a-72b4e68e7067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:26.715706 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1271/2508] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:26.728040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b4e15-f3a2-496c-8b8b-306c68245126 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:26.729858 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-360b4e15-f3a2-496c-8b8b-306c68245126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:26.806722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b4e15-f3a2-496c-8b8b-306c68245126 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:26.812845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7b4fb-222e-47ca-9118-4d993e2fd47c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:26.813551 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7b4fb-222e-47ca-9118-4d993e2fd47c service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:26.813837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7b4fb-222e-47ca-9118-4d993e2fd47c service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:26.814116 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7b4fb-222e-47ca-9118-4d993e2fd47c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:26.823825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7b4fb-222e-47ca-9118-4d993e2fd47c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:26.825529 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7b4fb-222e-47ca-9118-4d993e2fd47c service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:26.836551 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1238/2509] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:41:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:41:27.044646 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb97538-2752-4743-b77a-72b4e68e7067 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:27.046064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5fb97538-2752-4743-b77a-72b4e68e7067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:27.072429 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:27.109599 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb97538-2752-4743-b77a-72b4e68e7067 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:27.114377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b2ad23-854e-4028-a0d2-6e058350a4f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:27.114943 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2ad23-854e-4028-a0d2-6e058350a4f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:27.115225 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2ad23-854e-4028-a0d2-6e058350a4f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:27.115529 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b2ad23-854e-4028-a0d2-6e058350a4f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:27.117537 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-28b2ad23-854e-4028-a0d2-6e058350a4f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:27.184784 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b2ad23-854e-4028-a0d2-6e058350a4f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:27.186252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b2ad23-854e-4028-a0d2-6e058350a4f1 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:27.198478 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1272/2510] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:41:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:41:30.108461 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7b4fb-222e-47ca-9118-4d993e2fd47c service nova] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:30.121133 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7b4fb-222e-47ca-9118-4d993e2fd47c service nova] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:30.129444 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b253d6-4fe2-4ea7-91dd-416e00d874c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:30.129444 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b253d6-4fe2-4ea7-91dd-416e00d874c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:30.129444 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b253d6-4fe2-4ea7-91dd-416e00d874c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:30.129444 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b253d6-4fe2-4ea7-91dd-416e00d874c0 admin admin] PATH_INFO: `/v3/users/65fcedd910c6461d8d9a43efdef6154e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:30.130207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b253d6-4fe2-4ea7-91dd-416e00d874c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65fcedd910c6461d8d9a43efdef6154e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:30.131983 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b253d6-4fe2-4ea7-91dd-416e00d874c0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:30.364059 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-25b253d6-4fe2-4ea7-91dd-416e00d874c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65fcedd910c6461d8d9a43efdef6154e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:30.364959 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-25b253d6-4fe2-4ea7-91dd-416e00d874c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65fcedd910c6461d8d9a43efdef6154e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:30.546227 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-25b253d6-4fe2-4ea7-91dd-416e00d874c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65fcedd910c6461d8d9a43efdef6154e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:30.555480 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-25b253d6-4fe2-4ea7-91dd-416e00d874c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65fcedd910c6461d8d9a43efdef6154e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:30.556628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-25b253d6-4fe2-4ea7-91dd-416e00d874c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65fcedd910c6461d8d9a43efdef6154e'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:30.582307 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1239/2511] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:41:30 2022] DELETE /identity/v3/users/65fcedd910c6461d8d9a43efdef6154e => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:30.588114 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ca7ee-f39b-42d7-8070-b158ee5026d5 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:30.588653 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ca7ee-f39b-42d7-8070-b158ee5026d5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:30.588653 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ca7ee-f39b-42d7-8070-b158ee5026d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:30.611819 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-005ca7ee-f39b-42d7-8070-b158ee5026d5 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:30.613315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-005ca7ee-f39b-42d7-8070-b158ee5026d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:30.709895 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1273/2512] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:30.712302 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b253d6-4fe2-4ea7-91dd-416e00d874c0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:30.712302 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-25b253d6-4fe2-4ea7-91dd-416e00d874c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:30.741598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:30.811986 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b253d6-4fe2-4ea7-91dd-416e00d874c0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:30.830948 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3bf7e-703b-4fcf-b30c-4e4fcae7b865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:30.831704 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3bf7e-703b-4fcf-b30c-4e4fcae7b865 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:30.833096 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3bf7e-703b-4fcf-b30c-4e4fcae7b865 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:30.833320 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3bf7e-703b-4fcf-b30c-4e4fcae7b865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:30.850246 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc3bf7e-703b-4fcf-b30c-4e4fcae7b865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:30.851951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc3bf7e-703b-4fcf-b30c-4e4fcae7b865 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:30.864323 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1240/2513] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:41:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:41:31.030831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ca7ee-f39b-42d7-8070-b158ee5026d5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:31.032504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-005ca7ee-f39b-42d7-8070-b158ee5026d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:31.045329 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ca7ee-f39b-42d7-8070-b158ee5026d5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:31.050450 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc605784-21ef-4643-a950-cee6b0125c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:31.057863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc605784-21ef-4643-a950-cee6b0125c63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:31.057863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc605784-21ef-4643-a950-cee6b0125c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:31.057863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc605784-21ef-4643-a950-cee6b0125c63 admin admin] PATH_INFO: `/v3/users/18f2856b1d3742a1aca89b4aeec0a3bd` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:31.057863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc605784-21ef-4643-a950-cee6b0125c63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18f2856b1d3742a1aca89b4aeec0a3bd)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:31.057863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc605784-21ef-4643-a950-cee6b0125c63 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:31.333361 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-dc605784-21ef-4643-a950-cee6b0125c63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18f2856b1d3742a1aca89b4aeec0a3bd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:31.334039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-dc605784-21ef-4643-a950-cee6b0125c63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18f2856b1d3742a1aca89b4aeec0a3bd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:31.341281 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-dc605784-21ef-4643-a950-cee6b0125c63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18f2856b1d3742a1aca89b4aeec0a3bd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:31.437616 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-dc605784-21ef-4643-a950-cee6b0125c63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18f2856b1d3742a1aca89b4aeec0a3bd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:31.438268 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-dc605784-21ef-4643-a950-cee6b0125c63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18f2856b1d3742a1aca89b4aeec0a3bd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:31.440121 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:31.457584 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1274/2514] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:41:31 2022] DELETE /identity/v3/users/18f2856b1d3742a1aca89b4aeec0a3bd => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:41:31.513362 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3bf7e-703b-4fcf-b30c-4e4fcae7b865 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:31.516004 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6dc3bf7e-703b-4fcf-b30c-4e4fcae7b865 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:31.538364 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3bf7e-703b-4fcf-b30c-4e4fcae7b865 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:31.609497 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b20b54-479b-4981-a8d9-c635bef6b98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:31.610976 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b20b54-479b-4981-a8d9-c635bef6b98d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:31.611606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b20b54-479b-4981-a8d9-c635bef6b98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:31.612099 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b20b54-479b-4981-a8d9-c635bef6b98d admin admin] PATH_INFO: `/v3/projects/561fc307f26d40b1a73b287ff89293f1` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:31.614969 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b20b54-479b-4981-a8d9-c635bef6b98d admin admin] RBAC: Authorizing `identity:delete_project(project_id=561fc307f26d40b1a73b287ff89293f1)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:31.618793 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b20b54-479b-4981-a8d9-c635bef6b98d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:31.725429 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc605784-21ef-4643-a950-cee6b0125c63 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:31.731857 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-dc605784-21ef-4643-a950-cee6b0125c63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:31.741464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b0b20b54-479b-4981-a8d9-c635bef6b98d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '561fc307f26d40b1a73b287ff89293f1', 'request_id': 'req-b0b20b54-479b-4981-a8d9-c635bef6b98d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:31.741749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc605784-21ef-4643-a950-cee6b0125c63 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:31.749194 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b0b20b54-479b-4981-a8d9-c635bef6b98d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '561fc307f26d40b1a73b287ff89293f1', 'request_id': 'req-b0b20b54-479b-4981-a8d9-c635bef6b98d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:31.756903 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b0b20b54-479b-4981-a8d9-c635bef6b98d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '561fc307f26d40b1a73b287ff89293f1', 'request_id': 'req-b0b20b54-479b-4981-a8d9-c635bef6b98d'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:31.800978 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3a290d-e683-4d9e-9c17-834c7aa32d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:31.801786 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a290d-e683-4d9e-9c17-834c7aa32d8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:31.802067 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a290d-e683-4d9e-9c17-834c7aa32d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:31.802341 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3a290d-e683-4d9e-9c17-834c7aa32d8a admin admin] PATH_INFO: `/v3/projects/80a9bf240ed0447faaf33bec8f19cb70` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:31.803554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3a290d-e683-4d9e-9c17-834c7aa32d8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=80a9bf240ed0447faaf33bec8f19cb70)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:31.805469 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3a290d-e683-4d9e-9c17-834c7aa32d8a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:31.866028 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b0b20b54-479b-4981-a8d9-c635bef6b98d admin admin] The token cache is being invalidate because project 561fc307f26d40b1a73b287ff89293f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:41:31.866280 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b0b20b54-479b-4981-a8d9-c635bef6b98d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:31.866878 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b0b20b54-479b-4981-a8d9-c635bef6b98d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:31.868516 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1241/2515] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:41:31 2022] DELETE /identity/v3/projects/561fc307f26d40b1a73b287ff89293f1 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:31.873401 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76223073-e5d2-4c31-8dca-4cca59aab95f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:31.873698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76223073-e5d2-4c31-8dca-4cca59aab95f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:31.873982 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76223073-e5d2-4c31-8dca-4cca59aab95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:31.893819 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-76223073-e5d2-4c31-8dca-4cca59aab95f None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:31.896019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-76223073-e5d2-4c31-8dca-4cca59aab95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:31.903699 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1242/2516] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:41:31.911883 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1e1aa-4aa2-4503-9b21-777c9be3a845 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:31.912170 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1e1aa-4aa2-4503-9b21-777c9be3a845 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:31.912450 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b1e1aa-4aa2-4503-9b21-777c9be3a845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:31.931428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-78b1e1aa-4aa2-4503-9b21-777c9be3a845 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:31.932710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-78b1e1aa-4aa2-4503-9b21-777c9be3a845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:31.941267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1243/2517] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:41:31.946114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1e1aa-4aa2-4503-9b21-777c9be3a845 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:31.956080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b1e1aa-4aa2-4503-9b21-777c9be3a845 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:31.970178 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1c3a290d-e683-4d9e-9c17-834c7aa32d8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80a9bf240ed0447faaf33bec8f19cb70', 'request_id': 'req-1c3a290d-e683-4d9e-9c17-834c7aa32d8a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:31.987731 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:32.015835 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1e033-bd5f-4778-8a11-ca06f1e1288b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:32.016524 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1e033-bd5f-4778-8a11-ca06f1e1288b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:32.016803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1e033-bd5f-4778-8a11-ca06f1e1288b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:32.017075 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1e033-bd5f-4778-8a11-ca06f1e1288b admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:32.017790 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1e033-bd5f-4778-8a11-ca06f1e1288b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:41:32.018093 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1e033-bd5f-4778-8a11-ca06f1e1288b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:32.020435 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1e033-bd5f-4778-8a11-ca06f1e1288b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:32.032764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1244/2518] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:41:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:41:32.037885 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1e033-bd5f-4778-8a11-ca06f1e1288b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:32.048172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1e033-bd5f-4778-8a11-ca06f1e1288b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:32.052944 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490ebc2-df9b-47b3-bbb3-4cac4b5d4257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:32.053581 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490ebc2-df9b-47b3-bbb3-4cac4b5d4257 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:32.053865 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490ebc2-df9b-47b3-bbb3-4cac4b5d4257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:32.054180 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490ebc2-df9b-47b3-bbb3-4cac4b5d4257 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:32.054861 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d490ebc2-df9b-47b3-bbb3-4cac4b5d4257 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:32.056780 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d490ebc2-df9b-47b3-bbb3-4cac4b5d4257 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:32.071968 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1c3a290d-e683-4d9e-9c17-834c7aa32d8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80a9bf240ed0447faaf33bec8f19cb70', 'request_id': 'req-1c3a290d-e683-4d9e-9c17-834c7aa32d8a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:32.078788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1c3a290d-e683-4d9e-9c17-834c7aa32d8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80a9bf240ed0447faaf33bec8f19cb70', 'request_id': 'req-1c3a290d-e683-4d9e-9c17-834c7aa32d8a'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:32.092111 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1c3a290d-e683-4d9e-9c17-834c7aa32d8a admin admin] The token cache is being invalidate because project 80a9bf240ed0447faaf33bec8f19cb70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:41:32.092111 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1c3a290d-e683-4d9e-9c17-834c7aa32d8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:32.092786 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-1c3a290d-e683-4d9e-9c17-834c7aa32d8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:32.094922 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1275/2519] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:41:31 2022] DELETE /identity/v3/projects/80a9bf240ed0447faaf33bec8f19cb70 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:32.101225 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697b605-6ad1-41e3-b300-5390fa693f4a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:32.101516 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697b605-6ad1-41e3-b300-5390fa693f4a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:32.101802 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0697b605-6ad1-41e3-b300-5390fa693f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:32.121379 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0697b605-6ad1-41e3-b300-5390fa693f4a None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:32.122681 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0697b605-6ad1-41e3-b300-5390fa693f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:32.131152 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1276/2520] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:32.138679 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798abd30-5b8b-415b-b5ce-50671d962a67 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:32.138969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798abd30-5b8b-415b-b5ce-50671d962a67 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:32.139256 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798abd30-5b8b-415b-b5ce-50671d962a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:32.153548 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1245/2521] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:41:32 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:41:32.158685 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490ebc2-df9b-47b3-bbb3-4cac4b5d4257 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:32.159273 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-798abd30-5b8b-415b-b5ce-50671d962a67 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:32.160493 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-798abd30-5b8b-415b-b5ce-50671d962a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:32.160843 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d490ebc2-df9b-47b3-bbb3-4cac4b5d4257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:32.228040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490ebc2-df9b-47b3-bbb3-4cac4b5d4257 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:32.229031 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1277/2522] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:41:32.233413 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798abd30-5b8b-415b-b5ce-50671d962a67 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:32.235538 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644a0cd-d1ef-48fe-995a-38e52b1d05ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:32.236302 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644a0cd-d1ef-48fe-995a-38e52b1d05ec admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:32.236760 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644a0cd-d1ef-48fe-995a-38e52b1d05ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:32.237038 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3644a0cd-d1ef-48fe-995a-38e52b1d05ec admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:32.237736 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644a0cd-d1ef-48fe-995a-38e52b1d05ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:32.239172 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:32.240509 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3644a0cd-d1ef-48fe-995a-38e52b1d05ec admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:32.243164 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798abd30-5b8b-415b-b5ce-50671d962a67 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:32.249904 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ef79e-44ad-4e01-a50b-41673d102995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:32.256385 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ef79e-44ad-4e01-a50b-41673d102995 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:32.256385 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ef79e-44ad-4e01-a50b-41673d102995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:32.256385 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ef79e-44ad-4e01-a50b-41673d102995 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:32.256385 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ef79e-44ad-4e01-a50b-41673d102995 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:41:32.256385 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ef79e-44ad-4e01-a50b-41673d102995 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:32.256385 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95ef79e-44ad-4e01-a50b-41673d102995 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:32.270526 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1278/2523] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:41:32.273100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ef79e-44ad-4e01-a50b-41673d102995 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:32.283319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ef79e-44ad-4e01-a50b-41673d102995 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:32.288034 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69edc9-04a2-403d-9a32-d95a008a3958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:32.288683 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69edc9-04a2-403d-9a32-d95a008a3958 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:32.288970 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69edc9-04a2-403d-9a32-d95a008a3958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:32.289253 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69edc9-04a2-403d-9a32-d95a008a3958 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:32.289944 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69edc9-04a2-403d-9a32-d95a008a3958 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:32.291903 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69edc9-04a2-403d-9a32-d95a008a3958 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:32.432089 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1246/2524] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:41:32 2022] POST /identity/v3/users => generated 283 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:41:32.439211 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644a0cd-d1ef-48fe-995a-38e52b1d05ec admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:32.450689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3644a0cd-d1ef-48fe-995a-38e52b1d05ec admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:32.451198 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1279/2525] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:41:32 2022] POST /identity/v3/projects => generated 373 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:41:32.455255 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69edc9-04a2-403d-9a32-d95a008a3958 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:32.514706 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027095ef-b7b4-411d-bd87-27cf15a3a886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:32.515339 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027095ef-b7b4-411d-bd87-27cf15a3a886 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:32.515828 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027095ef-b7b4-411d-bd87-27cf15a3a886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:32.516142 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027095ef-b7b4-411d-bd87-27cf15a3a886 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:32.516895 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027095ef-b7b4-411d-bd87-27cf15a3a886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:32.518496 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027095ef-b7b4-411d-bd87-27cf15a3a886 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:32.521710 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69edc9-04a2-403d-9a32-d95a008a3958 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:32.526609 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716c0ce-37a4-4260-8835-e8198101cd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:32.531580 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716c0ce-37a4-4260-8835-e8198101cd6e admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:32.531580 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716c0ce-37a4-4260-8835-e8198101cd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:32.531580 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716c0ce-37a4-4260-8835-e8198101cd6e admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:32.531580 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4716c0ce-37a4-4260-8835-e8198101cd6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:32.532078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1247/2526] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:41:32.532201 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4716c0ce-37a4-4260-8835-e8198101cd6e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:32.533135 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027095ef-b7b4-411d-bd87-27cf15a3a886 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:32.544213 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027095ef-b7b4-411d-bd87-27cf15a3a886 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:32.549047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647aaac-0494-4213-bf00-1020826b2a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:32.549708 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647aaac-0494-4213-bf00-1020826b2a90 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:32.549991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647aaac-0494-4213-bf00-1020826b2a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:32.550303 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647aaac-0494-4213-bf00-1020826b2a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:32.551727 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c647aaac-0494-4213-bf00-1020826b2a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:32.553315 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c647aaac-0494-4213-bf00-1020826b2a90 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:32.563537 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1248/2527] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:41:32.568733 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647aaac-0494-4213-bf00-1020826b2a90 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:32.578188 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647aaac-0494-4213-bf00-1020826b2a90 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:32.582811 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1150c2-da6e-4a3d-b772-3aa62330ea90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:32.583785 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1150c2-da6e-4a3d-b772-3aa62330ea90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:32.584074 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1150c2-da6e-4a3d-b772-3aa62330ea90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:32.584353 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1150c2-da6e-4a3d-b772-3aa62330ea90 admin admin] PATH_INFO: `/v3/projects/2a02aa0af8ea43debd298f9b3dc98c3d/users/3cead99b97164aa988c61998897f947d/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:32.600061 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1150c2-da6e-4a3d-b772-3aa62330ea90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a02aa0af8ea43debd298f9b3dc98c3d, user_id=3cead99b97164aa988c61998897f947d, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:32.602828 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1150c2-da6e-4a3d-b772-3aa62330ea90 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:32.606032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:32.627733 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1280/2528] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:41:32 2022] POST /identity/v3/users => generated 287 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:41:32.632105 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716c0ce-37a4-4260-8835-e8198101cd6e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:32.642206 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716c0ce-37a4-4260-8835-e8198101cd6e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:32.651519 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bcd68-7cc4-4488-986d-ec7f9823483e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:32.652194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bcd68-7cc4-4488-986d-ec7f9823483e admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:32.652474 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bcd68-7cc4-4488-986d-ec7f9823483e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:32.652746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bcd68-7cc4-4488-986d-ec7f9823483e admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:32.653472 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526bcd68-7cc4-4488-986d-ec7f9823483e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:32.655070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526bcd68-7cc4-4488-986d-ec7f9823483e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:32.666188 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1281/2529] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:41:32.670486 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bcd68-7cc4-4488-986d-ec7f9823483e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:32.681704 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bcd68-7cc4-4488-986d-ec7f9823483e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:32.686774 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f9c33-72fe-4310-bda7-84c2a7a954f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:32.687438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f9c33-72fe-4310-bda7-84c2a7a954f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:32.687751 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f9c33-72fe-4310-bda7-84c2a7a954f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:32.688026 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f9c33-72fe-4310-bda7-84c2a7a954f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:32.688745 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f9c33-72fe-4310-bda7-84c2a7a954f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:32.690618 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f9c33-72fe-4310-bda7-84c2a7a954f2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:32.701884 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1282/2530] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:41:32.706513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f9c33-72fe-4310-bda7-84c2a7a954f2 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:32.718195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f9c33-72fe-4310-bda7-84c2a7a954f2 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:32.722979 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1249/2531] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:41:32 2022] PUT /identity/v3/projects/2a02aa0af8ea43debd298f9b3dc98c3d/users/3cead99b97164aa988c61998897f947d/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:41:32.735084 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17831f-8dca-4b75-8815-59d32a65b52a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:32.735365 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17831f-8dca-4b75-8815-59d32a65b52a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:32.735700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17831f-8dca-4b75-8815-59d32a65b52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:32.764633 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-0d17831f-8dca-4b75-8815-59d32a65b52a None None] MFA Rules not processed for user `3cead99b97164aa988c61998897f947d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:32.765926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0d17831f-8dca-4b75-8815-59d32a65b52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:32.780294 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08ab75-4c81-4c60-809a-e48cce9955b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:32.781183 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08ab75-4c81-4c60-809a-e48cce9955b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:32.781459 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08ab75-4c81-4c60-809a-e48cce9955b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:32.781732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08ab75-4c81-4c60-809a-e48cce9955b8 admin admin] PATH_INFO: `/v3/projects/4e825a3b75e24dab8e18fb5754654360/users/abe99ba29de64fc0b03b21823887e5b8/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:32.797495 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08ab75-4c81-4c60-809a-e48cce9955b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e825a3b75e24dab8e18fb5754654360, user_id=abe99ba29de64fc0b03b21823887e5b8, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:32.800226 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08ab75-4c81-4c60-809a-e48cce9955b8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:32.837586 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1250/2532] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:32 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:41:32.842270 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17831f-8dca-4b75-8815-59d32a65b52a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:32.853588 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17831f-8dca-4b75-8815-59d32a65b52a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:32.858099 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4facf-d992-4ed0-a413-85dc51c7c557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:32.858749 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4facf-d992-4ed0-a413-85dc51c7c557 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:32.859022 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4facf-d992-4ed0-a413-85dc51c7c557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:32.859301 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b4facf-d992-4ed0-a413-85dc51c7c557 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:32.860027 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4facf-d992-4ed0-a413-85dc51c7c557 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:32.861936 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b4facf-d992-4ed0-a413-85dc51c7c557 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:32.912059 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1283/2533] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:41:32 2022] PUT /identity/v3/projects/4e825a3b75e24dab8e18fb5754654360/users/abe99ba29de64fc0b03b21823887e5b8/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:41:32.923143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd029c8d-57f5-49d2-b617-4c9a6ccda778 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:32.923621 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd029c8d-57f5-49d2-b617-4c9a6ccda778 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:32.924000 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd029c8d-57f5-49d2-b617-4c9a6ccda778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:32.953892 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-dd029c8d-57f5-49d2-b617-4c9a6ccda778 None None] MFA Rules not processed for user `abe99ba29de64fc0b03b21823887e5b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:32.955177 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-dd029c8d-57f5-49d2-b617-4c9a6ccda778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:32.964413 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:33.016409 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1251/2534] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:41:32 2022] POST /identity/v3/projects => generated 365 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:41:33.021339 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4facf-d992-4ed0-a413-85dc51c7c557 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:33.023832 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1284/2535] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:41:33.035168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd029c8d-57f5-49d2-b617-4c9a6ccda778 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:33.037065 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-dd029c8d-57f5-49d2-b617-4c9a6ccda778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:33.096579 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b4facf-d992-4ed0-a413-85dc51c7c557 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:33.101710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda7d97-21de-4452-a8a6-284040a7c962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:33.102429 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda7d97-21de-4452-a8a6-284040a7c962 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:33.102715 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda7d97-21de-4452-a8a6-284040a7c962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:33.102993 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda7d97-21de-4452-a8a6-284040a7c962 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:33.103729 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda7d97-21de-4452-a8a6-284040a7c962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:33.105713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbda7d97-21de-4452-a8a6-284040a7c962 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:33.123041 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd029c8d-57f5-49d2-b617-4c9a6ccda778 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:33.193526 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4364dd6-45a2-49f1-8287-8905765f5d96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:33.194191 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4364dd6-45a2-49f1-8287-8905765f5d96 service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:33.194478 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4364dd6-45a2-49f1-8287-8905765f5d96 service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:33.194762 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4364dd6-45a2-49f1-8287-8905765f5d96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:33.204087 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1252/2536] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:41:33 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:41:33.209127 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda7d97-21de-4452-a8a6-284040a7c962 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:33.219347 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda7d97-21de-4452-a8a6-284040a7c962 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:33.225075 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e67484d-09db-44c6-9cfd-ec227a4535b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:33.225756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67484d-09db-44c6-9cfd-ec227a4535b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:33.226042 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67484d-09db-44c6-9cfd-ec227a4535b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:33.226317 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e67484d-09db-44c6-9cfd-ec227a4535b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:33.227048 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e67484d-09db-44c6-9cfd-ec227a4535b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:33.228996 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e67484d-09db-44c6-9cfd-ec227a4535b7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:33.246632 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:33.253990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1253/2537] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:41:33.256715 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e67484d-09db-44c6-9cfd-ec227a4535b7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:33.269103 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4364dd6-45a2-49f1-8287-8905765f5d96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:33.272434 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4364dd6-45a2-49f1-8287-8905765f5d96 service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:33.273661 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e67484d-09db-44c6-9cfd-ec227a4535b7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:33.279486 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27eee4-6954-4690-9861-d61873d9a60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:33.280439 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27eee4-6954-4690-9861-d61873d9a60b admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:33.280695 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27eee4-6954-4690-9861-d61873d9a60b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:33.280972 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27eee4-6954-4690-9861-d61873d9a60b admin admin] PATH_INFO: `/v3/projects/f96bc9295bd64603a6e4a41fc3fe4670/users/3f2cadc3a8af42848a18a46e2bd9183f/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:33.285983 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1285/2538] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:41:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:41:33.298750 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27eee4-6954-4690-9861-d61873d9a60b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96bc9295bd64603a6e4a41fc3fe4670, user_id=3f2cadc3a8af42848a18a46e2bd9183f, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:33.302163 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27eee4-6954-4690-9861-d61873d9a60b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:33.409628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1254/2539] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:41:33 2022] PUT /identity/v3/projects/f96bc9295bd64603a6e4a41fc3fe4670/users/3f2cadc3a8af42848a18a46e2bd9183f/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:41:33.415651 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4364dd6-45a2-49f1-8287-8905765f5d96 service nova] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:33.437996 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4364dd6-45a2-49f1-8287-8905765f5d96 service nova] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:33.552357 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27eee4-6954-4690-9861-d61873d9a60b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:33.552871 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df715c-aac1-4ac9-8e12-4d312ba427a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:33.553662 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df715c-aac1-4ac9-8e12-4d312ba427a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:33.553783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df715c-aac1-4ac9-8e12-4d312ba427a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:33.553975 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df715c-aac1-4ac9-8e12-4d312ba427a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:33.554732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df715c-aac1-4ac9-8e12-4d312ba427a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:33.556466 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df715c-aac1-4ac9-8e12-4d312ba427a9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:33.563686 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27eee4-6954-4690-9861-d61873d9a60b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:33.569033 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1286/2540] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:33 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:41:33.570750 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f9aa1-4c97-4e9d-a21a-86814a70dc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:33.571755 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f9aa1-4c97-4e9d-a21a-86814a70dc45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:33.572161 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f9aa1-4c97-4e9d-a21a-86814a70dc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:33.572467 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f9aa1-4c97-4e9d-a21a-86814a70dc45 admin admin] PATH_INFO: `/v3/users/abe99ba29de64fc0b03b21823887e5b8` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:33.573674 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df715c-aac1-4ac9-8e12-4d312ba427a9 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:33.573800 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f9aa1-4c97-4e9d-a21a-86814a70dc45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abe99ba29de64fc0b03b21823887e5b8)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:33.575794 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f9aa1-4c97-4e9d-a21a-86814a70dc45 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:33.586821 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df715c-aac1-4ac9-8e12-4d312ba427a9 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:33.591644 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f0016e-62f9-4877-8e0e-cf4b2701af21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:33.592536 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0016e-62f9-4877-8e0e-cf4b2701af21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:33.592817 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0016e-62f9-4877-8e0e-cf4b2701af21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:33.593096 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f0016e-62f9-4877-8e0e-cf4b2701af21 admin admin] PATH_INFO: `/v3/projects/f96bc9295bd64603a6e4a41fc3fe4670/users/3f2cadc3a8af42848a18a46e2bd9183f/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:33.595216 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f0016e-62f9-4877-8e0e-cf4b2701af21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96bc9295bd64603a6e4a41fc3fe4670, user_id=3f2cadc3a8af42848a18a46e2bd9183f, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:33.597550 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:33.598949 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f0016e-62f9-4877-8e0e-cf4b2701af21 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:33.609719 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1287/2541] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:41:33 2022] PUT /identity/v3/projects/f96bc9295bd64603a6e4a41fc3fe4670/users/3f2cadc3a8af42848a18a46e2bd9183f/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:33.628674 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed690d0-fe25-4a49-9459-8271eb25bf2d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:33.629009 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed690d0-fe25-4a49-9459-8271eb25bf2d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:33.629980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed690d0-fe25-4a49-9459-8271eb25bf2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:33.657332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-aed690d0-fe25-4a49-9459-8271eb25bf2d None None] MFA Rules not processed for user `3f2cadc3a8af42848a18a46e2bd9183f`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:33.658530 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-aed690d0-fe25-4a49-9459-8271eb25bf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:33.741403 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1288/2542] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:33 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:41:33.753195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed690d0-fe25-4a49-9459-8271eb25bf2d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:33.754602 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-aed690d0-fe25-4a49-9459-8271eb25bf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:33.813143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed690d0-fe25-4a49-9459-8271eb25bf2d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:33.817424 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620bc85-941e-440f-8852-544ff2d4ede9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:33.818273 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620bc85-941e-440f-8852-544ff2d4ede9 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:33.818554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620bc85-941e-440f-8852-544ff2d4ede9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:33.818887 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620bc85-941e-440f-8852-544ff2d4ede9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:33.826873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:33.827111 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ac1f9aa1-4c97-4e9d-a21a-86814a70dc45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abe99ba29de64fc0b03b21823887e5b8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:33.827937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ac1f9aa1-4c97-4e9d-a21a-86814a70dc45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abe99ba29de64fc0b03b21823887e5b8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:33.829672 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9620bc85-941e-440f-8852-544ff2d4ede9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:33.831158 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9620bc85-941e-440f-8852-544ff2d4ede9 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:33.841849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1289/2543] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:41:33 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:41:33.939356 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ac1f9aa1-4c97-4e9d-a21a-86814a70dc45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abe99ba29de64fc0b03b21823887e5b8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:33.948315 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ac1f9aa1-4c97-4e9d-a21a-86814a70dc45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abe99ba29de64fc0b03b21823887e5b8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:33.949030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ac1f9aa1-4c97-4e9d-a21a-86814a70dc45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abe99ba29de64fc0b03b21823887e5b8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:33.976465 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1255/2544] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:41:33 2022] DELETE /identity/v3/users/abe99ba29de64fc0b03b21823887e5b8 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:41:33.980252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5b323-18d2-45bc-97b7-b29d3d4b043d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:33.980625 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5b323-18d2-45bc-97b7-b29d3d4b043d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:33.980970 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5b323-18d2-45bc-97b7-b29d3d4b043d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:34.007856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-b8c5b323-18d2-45bc-97b7-b29d3d4b043d None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:34.007856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b8c5b323-18d2-45bc-97b7-b29d3d4b043d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:34.078128 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1290/2545] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:41:34.092470 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f9aa1-4c97-4e9d-a21a-86814a70dc45 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:34.094206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ac1f9aa1-4c97-4e9d-a21a-86814a70dc45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:34.097263 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5b323-18d2-45bc-97b7-b29d3d4b043d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:34.098615 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b8c5b323-18d2-45bc-97b7-b29d3d4b043d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:34.163789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f9aa1-4c97-4e9d-a21a-86814a70dc45 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:34.166407 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5b323-18d2-45bc-97b7-b29d3d4b043d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:34.167628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddda92e-d27b-4834-9649-de797bad2324 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:34.168416 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddda92e-d27b-4834-9649-de797bad2324 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:34.168706 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddda92e-d27b-4834-9649-de797bad2324 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:34.168985 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddda92e-d27b-4834-9649-de797bad2324 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:34.170130 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efd91dd-02c3-460c-971c-ce6290cf802e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:34.170768 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd91dd-02c3-460c-971c-ce6290cf802e service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:34.171080 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd91dd-02c3-460c-971c-ce6290cf802e service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:34.171318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efd91dd-02c3-460c-971c-ce6290cf802e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:34.173202 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8efd91dd-02c3-460c-971c-ce6290cf802e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:34.179519 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddda92e-d27b-4834-9649-de797bad2324 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:34.180932 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddda92e-d27b-4834-9649-de797bad2324 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:34.191875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1256/2546] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:41:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:41:34.237752 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd91dd-02c3-460c-971c-ce6290cf802e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:34.239223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efd91dd-02c3-460c-971c-ce6290cf802e service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:34.250315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1291/2547] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:41:34 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:41:34.640902 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddda92e-d27b-4834-9649-de797bad2324 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:34.642448 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-eddda92e-d27b-4834-9649-de797bad2324 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:34.656952 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddda92e-d27b-4834-9649-de797bad2324 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:34.663750 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa0af1-98cf-43d2-98fe-2ef6654dfde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:34.664904 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa0af1-98cf-43d2-98fe-2ef6654dfde1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:34.665546 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa0af1-98cf-43d2-98fe-2ef6654dfde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:34.666042 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa0af1-98cf-43d2-98fe-2ef6654dfde1 admin admin] PATH_INFO: `/v3/projects/4e825a3b75e24dab8e18fb5754654360` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:34.667684 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa0af1-98cf-43d2-98fe-2ef6654dfde1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e825a3b75e24dab8e18fb5754654360)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:34.672856 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa0af1-98cf-43d2-98fe-2ef6654dfde1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:34.828579 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a8aa0af1-98cf-43d2-98fe-2ef6654dfde1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e825a3b75e24dab8e18fb5754654360', 'request_id': 'req-a8aa0af1-98cf-43d2-98fe-2ef6654dfde1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:34.843818 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a8aa0af1-98cf-43d2-98fe-2ef6654dfde1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e825a3b75e24dab8e18fb5754654360', 'request_id': 'req-a8aa0af1-98cf-43d2-98fe-2ef6654dfde1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:34.871034 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a8aa0af1-98cf-43d2-98fe-2ef6654dfde1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e825a3b75e24dab8e18fb5754654360', 'request_id': 'req-a8aa0af1-98cf-43d2-98fe-2ef6654dfde1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:34.986357 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a8aa0af1-98cf-43d2-98fe-2ef6654dfde1 admin admin] The token cache is being invalidate because project 4e825a3b75e24dab8e18fb5754654360 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:41:34.986561 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a8aa0af1-98cf-43d2-98fe-2ef6654dfde1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:34.987221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-a8aa0af1-98cf-43d2-98fe-2ef6654dfde1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:34.988820 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1257/2548] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:41:34 2022] DELETE /identity/v3/projects/4e825a3b75e24dab8e18fb5754654360 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:34.994741 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a2cb6-9fa2-450e-bdd8-ada99c7b1627 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:34.995228 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a2cb6-9fa2-450e-bdd8-ada99c7b1627 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:34.995419 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a2cb6-9fa2-450e-bdd8-ada99c7b1627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:35.015025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-0a8a2cb6-9fa2-450e-bdd8-ada99c7b1627 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:35.016315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0a8a2cb6-9fa2-450e-bdd8-ada99c7b1627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:35.062747 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:35.078226 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1292/2549] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:35.087452 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931720a-f330-40ba-bda4-8ec220309d64 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:35.087773 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931720a-f330-40ba-bda4-8ec220309d64 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:35.088056 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931720a-f330-40ba-bda4-8ec220309d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:35.109373 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2931720a-f330-40ba-bda4-8ec220309d64 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:35.110603 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2931720a-f330-40ba-bda4-8ec220309d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:35.121667 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1258/2550] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:35.126016 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a2cb6-9fa2-450e-bdd8-ada99c7b1627 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:35.138336 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a2cb6-9fa2-450e-bdd8-ada99c7b1627 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:35.143815 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa728eb0-9101-4b39-b8d9-50ac73944916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:35.144472 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa728eb0-9101-4b39-b8d9-50ac73944916 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:35.144790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa728eb0-9101-4b39-b8d9-50ac73944916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:35.145146 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa728eb0-9101-4b39-b8d9-50ac73944916 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:35.146665 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa728eb0-9101-4b39-b8d9-50ac73944916 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:41:35.147651 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa728eb0-9101-4b39-b8d9-50ac73944916 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:35.150501 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa728eb0-9101-4b39-b8d9-50ac73944916 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:35.163595 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1293/2551] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:41:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:41:35.167920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931720a-f330-40ba-bda4-8ec220309d64 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:35.178130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931720a-f330-40ba-bda4-8ec220309d64 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:35.183511 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b01b8-af2a-4f43-8501-e185186d6412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:35.184392 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b01b8-af2a-4f43-8501-e185186d6412 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:35.184680 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b01b8-af2a-4f43-8501-e185186d6412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:35.185023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b01b8-af2a-4f43-8501-e185186d6412 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:35.185698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b01b8-af2a-4f43-8501-e185186d6412 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:35.187675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b01b8-af2a-4f43-8501-e185186d6412 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:35.192316 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:35.331643 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1259/2552] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:41:35 2022] POST /identity/v3/projects => generated 363 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:41:35.336901 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa728eb0-9101-4b39-b8d9-50ac73944916 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:35.347090 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa728eb0-9101-4b39-b8d9-50ac73944916 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:35.407986 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734b04f-7488-45cd-aed0-242170a2c533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:35.408701 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734b04f-7488-45cd-aed0-242170a2c533 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:35.409016 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734b04f-7488-45cd-aed0-242170a2c533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:35.409313 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734b04f-7488-45cd-aed0-242170a2c533 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:35.410095 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734b04f-7488-45cd-aed0-242170a2c533 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:35.412794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734b04f-7488-45cd-aed0-242170a2c533 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:35.506976 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1294/2553] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:41:35 2022] POST /identity/v3/users => generated 282 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:41:35.511321 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b01b8-af2a-4f43-8501-e185186d6412 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:35.522346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b01b8-af2a-4f43-8501-e185186d6412 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:35.530187 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294422dc-30da-4747-88e3-59fdb52f7f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:35.530912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294422dc-30da-4747-88e3-59fdb52f7f57 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:35.531194 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294422dc-30da-4747-88e3-59fdb52f7f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:35.531774 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294422dc-30da-4747-88e3-59fdb52f7f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:35.532336 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294422dc-30da-4747-88e3-59fdb52f7f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:35.534256 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294422dc-30da-4747-88e3-59fdb52f7f57 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:35.547071 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1260/2554] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:41:35.552736 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734b04f-7488-45cd-aed0-242170a2c533 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:35.563652 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734b04f-7488-45cd-aed0-242170a2c533 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:35.569860 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8d96d-d011-46a4-86b1-8d989a81800f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:35.570566 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8d96d-d011-46a4-86b1-8d989a81800f admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:35.571829 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8d96d-d011-46a4-86b1-8d989a81800f admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:35.571829 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8d96d-d011-46a4-86b1-8d989a81800f admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:35.572017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b8d96d-d011-46a4-86b1-8d989a81800f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:35.573584 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b8d96d-d011-46a4-86b1-8d989a81800f admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:35.590093 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1295/2555] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:41:35.590402 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294422dc-30da-4747-88e3-59fdb52f7f57 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:35.601256 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294422dc-30da-4747-88e3-59fdb52f7f57 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:35.606094 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b6611-c1cb-4387-b753-427c495d1b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:35.607004 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b6611-c1cb-4387-b753-427c495d1b6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:35.607291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b6611-c1cb-4387-b753-427c495d1b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:35.607607 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b6611-c1cb-4387-b753-427c495d1b6d admin admin] PATH_INFO: `/v3/projects/213b521c1c16457f80a61f0dd1309dff/users/cbe052cd17644afcaf854af8960857a2/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:35.623782 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b6611-c1cb-4387-b753-427c495d1b6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=213b521c1c16457f80a61f0dd1309dff, user_id=cbe052cd17644afcaf854af8960857a2, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:35.626481 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b6611-c1cb-4387-b753-427c495d1b6d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:35.696875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1261/2556] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:41:35 2022] PUT /identity/v3/projects/213b521c1c16457f80a61f0dd1309dff/users/cbe052cd17644afcaf854af8960857a2/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:35.707090 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad7dbd-02b5-4933-be66-357981716b5b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:35.707409 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad7dbd-02b5-4933-be66-357981716b5b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:35.707721 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad7dbd-02b5-4933-be66-357981716b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:35.735983 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-31ad7dbd-02b5-4933-be66-357981716b5b None None] MFA Rules not processed for user `cbe052cd17644afcaf854af8960857a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:35.737790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-31ad7dbd-02b5-4933-be66-357981716b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:35.829488 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1296/2557] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:35 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:35.837075 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b6611-c1cb-4387-b753-427c495d1b6d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:35.920256 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b6611-c1cb-4387-b753-427c495d1b6d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:35.925776 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd631d-781b-4898-8657-910f45d18a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:35.926483 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd631d-781b-4898-8657-910f45d18a3c admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:35.926774 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd631d-781b-4898-8657-910f45d18a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:35.927130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd631d-781b-4898-8657-910f45d18a3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:35.928023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd631d-781b-4898-8657-910f45d18a3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:35.929901 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd631d-781b-4898-8657-910f45d18a3c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:36.053970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1262/2558] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:41:35 2022] POST /identity/v3/projects => generated 365 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:41:36.059717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad7dbd-02b5-4933-be66-357981716b5b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:36.073717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad7dbd-02b5-4933-be66-357981716b5b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:36.128174 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:36.142881 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27038ff6-f2b7-42d0-a864-8d37d5585847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:36.143602 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27038ff6-f2b7-42d0-a864-8d37d5585847 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:36.146099 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27038ff6-f2b7-42d0-a864-8d37d5585847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:36.146099 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27038ff6-f2b7-42d0-a864-8d37d5585847 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:36.146099 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27038ff6-f2b7-42d0-a864-8d37d5585847 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:36.146901 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27038ff6-f2b7-42d0-a864-8d37d5585847 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:36.252080 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1297/2559] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:41:36 2022] POST /identity/v3/users => generated 289 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:41:36.257332 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd631d-781b-4898-8657-910f45d18a3c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:36.268860 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd631d-781b-4898-8657-910f45d18a3c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:36.274029 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f9064-cf0e-49e1-a3df-08fba63d043e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:36.274711 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f9064-cf0e-49e1-a3df-08fba63d043e admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:36.274992 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f9064-cf0e-49e1-a3df-08fba63d043e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:36.275269 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f9064-cf0e-49e1-a3df-08fba63d043e admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:36.276156 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f9064-cf0e-49e1-a3df-08fba63d043e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:36.277775 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36f9064-cf0e-49e1-a3df-08fba63d043e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:36.289873 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1263/2560] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:41:36.300506 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27038ff6-f2b7-42d0-a864-8d37d5585847 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:36.312260 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27038ff6-f2b7-42d0-a864-8d37d5585847 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:36.319148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9830d-3ac4-471f-a459-116b52f64fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:36.321671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9830d-3ac4-471f-a459-116b52f64fb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:36.321671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9830d-3ac4-471f-a459-116b52f64fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:36.321671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea9830d-3ac4-471f-a459-116b52f64fb9 admin admin] PATH_INFO: `/v3/projects/b923435d156a4759a380d8800628ade2/users/defc1bf5a4914160b39d9aabe5edb9cf/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:36.364126 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9830d-3ac4-471f-a459-116b52f64fb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b923435d156a4759a380d8800628ade2, user_id=defc1bf5a4914160b39d9aabe5edb9cf, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:36.367897 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea9830d-3ac4-471f-a459-116b52f64fb9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:36.570881 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1298/2561] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:41:36 2022] PUT /identity/v3/projects/b923435d156a4759a380d8800628ade2/users/defc1bf5a4914160b39d9aabe5edb9cf/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:41:36.574987 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f9064-cf0e-49e1-a3df-08fba63d043e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:36.585608 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f9064-cf0e-49e1-a3df-08fba63d043e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:36.659396 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd3e64b-63e9-48d9-93f4-2a9a6411695b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:36.660193 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3e64b-63e9-48d9-93f4-2a9a6411695b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:36.660550 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3e64b-63e9-48d9-93f4-2a9a6411695b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:36.660897 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd3e64b-63e9-48d9-93f4-2a9a6411695b admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:36.661721 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd3e64b-63e9-48d9-93f4-2a9a6411695b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:36.664300 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd3e64b-63e9-48d9-93f4-2a9a6411695b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:36.675688 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1264/2562] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:41:36.680500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9830d-3ac4-471f-a459-116b52f64fb9 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:36.691654 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea9830d-3ac4-471f-a459-116b52f64fb9 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:36.696704 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b065e1-4567-4c52-944b-7e09b5f43eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:36.697642 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b065e1-4567-4c52-944b-7e09b5f43eda admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:36.697928 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b065e1-4567-4c52-944b-7e09b5f43eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:36.698206 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b065e1-4567-4c52-944b-7e09b5f43eda admin admin] PATH_INFO: `/v3/projects/b923435d156a4759a380d8800628ade2/users/defc1bf5a4914160b39d9aabe5edb9cf/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:36.700292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b065e1-4567-4c52-944b-7e09b5f43eda admin admin] RBAC: Authorizing `identity:create_grant(project_id=b923435d156a4759a380d8800628ade2, user_id=defc1bf5a4914160b39d9aabe5edb9cf, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:36.702974 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b065e1-4567-4c52-944b-7e09b5f43eda admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:36.714238 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1299/2563] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:41:36 2022] PUT /identity/v3/projects/b923435d156a4759a380d8800628ade2/users/defc1bf5a4914160b39d9aabe5edb9cf/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:36.724257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d0164-de7a-4260-8722-fc18e12ca2b6 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:36.725269 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d0164-de7a-4260-8722-fc18e12ca2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:36.725394 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d0164-de7a-4260-8722-fc18e12ca2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:36.755299 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1e4d0164-de7a-4260-8722-fc18e12ca2b6 None None] MFA Rules not processed for user `defc1bf5a4914160b39d9aabe5edb9cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:36.757085 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1e4d0164-de7a-4260-8722-fc18e12ca2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:36.768865 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:36.851776 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1265/2564] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:36 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:41:36.864658 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b065e1-4567-4c52-944b-7e09b5f43eda admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:36.866625 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b2b065e1-4567-4c52-944b-7e09b5f43eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:36.977366 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b065e1-4567-4c52-944b-7e09b5f43eda admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:36.987265 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a7c9d-f50b-48d5-9ca1-891ab0d17006 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:36.987974 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a7c9d-f50b-48d5-9ca1-891ab0d17006 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:36.988263 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a7c9d-f50b-48d5-9ca1-891ab0d17006 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:36.988526 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a7c9d-f50b-48d5-9ca1-891ab0d17006 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:37.018446 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42a7c9d-f50b-48d5-9ca1-891ab0d17006 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:37.020051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42a7c9d-f50b-48d5-9ca1-891ab0d17006 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:37.037832 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1300/2565] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:41:36 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:41:38.421111 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d0164-de7a-4260-8722-fc18e12ca2b6 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:38.487284 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d0164-de7a-4260-8722-fc18e12ca2b6 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:38.492848 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f0e42-26cd-4165-982e-7e9d1d947942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:38.493722 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f0e42-26cd-4165-982e-7e9d1d947942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:38.494014 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f0e42-26cd-4165-982e-7e9d1d947942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:38.494292 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f0e42-26cd-4165-982e-7e9d1d947942 admin admin] PATH_INFO: `/v3/users/cbe052cd17644afcaf854af8960857a2` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:38.495577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f0e42-26cd-4165-982e-7e9d1d947942 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbe052cd17644afcaf854af8960857a2)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:38.497593 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f0e42-26cd-4165-982e-7e9d1d947942 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:38.698943 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-735f0e42-26cd-4165-982e-7e9d1d947942 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbe052cd17644afcaf854af8960857a2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:38.699488 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-735f0e42-26cd-4165-982e-7e9d1d947942 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbe052cd17644afcaf854af8960857a2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:38.797879 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-735f0e42-26cd-4165-982e-7e9d1d947942 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbe052cd17644afcaf854af8960857a2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:38.807313 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-735f0e42-26cd-4165-982e-7e9d1d947942 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbe052cd17644afcaf854af8960857a2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:38.808070 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-735f0e42-26cd-4165-982e-7e9d1d947942 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbe052cd17644afcaf854af8960857a2'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:38.829794 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1266/2566] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:41:38 2022] DELETE /identity/v3/users/cbe052cd17644afcaf854af8960857a2 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:38.838553 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33568ef6-6208-4476-ae28-884538952b70 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:38.838553 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33568ef6-6208-4476-ae28-884538952b70 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:38.838553 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33568ef6-6208-4476-ae28-884538952b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:38.854030 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-33568ef6-6208-4476-ae28-884538952b70 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:38.855291 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-33568ef6-6208-4476-ae28-884538952b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:38.921582 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1301/2567] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:38.932692 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f0e42-26cd-4165-982e-7e9d1d947942 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:38.933680 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-735f0e42-26cd-4165-982e-7e9d1d947942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:38.972532 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:39.001672 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f0e42-26cd-4165-982e-7e9d1d947942 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:39.006732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f062263-cf52-4376-bd72-3fe0be1b6dd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:39.007458 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f062263-cf52-4376-bd72-3fe0be1b6dd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:39.007817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f062263-cf52-4376-bd72-3fe0be1b6dd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:39.008127 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f062263-cf52-4376-bd72-3fe0be1b6dd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:39.017988 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f062263-cf52-4376-bd72-3fe0be1b6dd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:39.019432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f062263-cf52-4376-bd72-3fe0be1b6dd2 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:39.032907 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1267/2568] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:41:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:41:39.547969 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33568ef6-6208-4476-ae28-884538952b70 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:39.549468 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-33568ef6-6208-4476-ae28-884538952b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:39.562345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33568ef6-6208-4476-ae28-884538952b70 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:39.568995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598256b0-8e7b-4103-9a1b-d6dea818b476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:39.569700 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598256b0-8e7b-4103-9a1b-d6dea818b476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:39.569985 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598256b0-8e7b-4103-9a1b-d6dea818b476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:39.571562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598256b0-8e7b-4103-9a1b-d6dea818b476 admin admin] PATH_INFO: `/v3/projects/213b521c1c16457f80a61f0dd1309dff` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:39.571675 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598256b0-8e7b-4103-9a1b-d6dea818b476 admin admin] RBAC: Authorizing `identity:delete_project(project_id=213b521c1c16457f80a61f0dd1309dff)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:39.573571 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598256b0-8e7b-4103-9a1b-d6dea818b476 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:39.687418 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:39.698912 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-598256b0-8e7b-4103-9a1b-d6dea818b476 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '213b521c1c16457f80a61f0dd1309dff', 'request_id': 'req-598256b0-8e7b-4103-9a1b-d6dea818b476'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:39.797833 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-598256b0-8e7b-4103-9a1b-d6dea818b476 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '213b521c1c16457f80a61f0dd1309dff', 'request_id': 'req-598256b0-8e7b-4103-9a1b-d6dea818b476'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:39.805437 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-598256b0-8e7b-4103-9a1b-d6dea818b476 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '213b521c1c16457f80a61f0dd1309dff', 'request_id': 'req-598256b0-8e7b-4103-9a1b-d6dea818b476'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:39.816017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-598256b0-8e7b-4103-9a1b-d6dea818b476 admin admin] The token cache is being invalidate because project 213b521c1c16457f80a61f0dd1309dff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:41:39.816311 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-598256b0-8e7b-4103-9a1b-d6dea818b476 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:39.816958 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-598256b0-8e7b-4103-9a1b-d6dea818b476 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:39.819691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1302/2569] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:41:39 2022] DELETE /identity/v3/projects/213b521c1c16457f80a61f0dd1309dff => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:39.823920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f062263-cf52-4376-bd72-3fe0be1b6dd2 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:39.835463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f062263-cf52-4376-bd72-3fe0be1b6dd2 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:39.900585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373a9d2b-4b06-4558-b358-d2edb3721c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:39.901493 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a9d2b-4b06-4558-b358-d2edb3721c6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:39.901773 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a9d2b-4b06-4558-b358-d2edb3721c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:39.902047 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a9d2b-4b06-4558-b358-d2edb3721c6d admin admin] PATH_INFO: `/v3/users/defc1bf5a4914160b39d9aabe5edb9cf` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:39.903251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a9d2b-4b06-4558-b358-d2edb3721c6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=defc1bf5a4914160b39d9aabe5edb9cf)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:39.905431 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373a9d2b-4b06-4558-b358-d2edb3721c6d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:40.137514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-373a9d2b-4b06-4558-b358-d2edb3721c6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'defc1bf5a4914160b39d9aabe5edb9cf'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:40.138172 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-373a9d2b-4b06-4558-b358-d2edb3721c6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'defc1bf5a4914160b39d9aabe5edb9cf'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:40.249331 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-373a9d2b-4b06-4558-b358-d2edb3721c6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'defc1bf5a4914160b39d9aabe5edb9cf'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:40.257968 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-373a9d2b-4b06-4558-b358-d2edb3721c6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'defc1bf5a4914160b39d9aabe5edb9cf'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:40.258776 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-373a9d2b-4b06-4558-b358-d2edb3721c6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'defc1bf5a4914160b39d9aabe5edb9cf'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:40.277507 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1268/2570] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:41:39 2022] DELETE /identity/v3/users/defc1bf5a4914160b39d9aabe5edb9cf => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:40.845395 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598256b0-8e7b-4103-9a1b-d6dea818b476 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:40.846978 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-598256b0-8e7b-4103-9a1b-d6dea818b476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:40.860752 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598256b0-8e7b-4103-9a1b-d6dea818b476 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:40.921210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a17a20-2189-4716-bcb7-f959fae4e76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:40.922214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a17a20-2189-4716-bcb7-f959fae4e76b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:40.922470 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a17a20-2189-4716-bcb7-f959fae4e76b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:40.922745 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a17a20-2189-4716-bcb7-f959fae4e76b admin admin] PATH_INFO: `/v3/projects/b923435d156a4759a380d8800628ade2` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:40.924017 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a17a20-2189-4716-bcb7-f959fae4e76b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b923435d156a4759a380d8800628ade2)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:40.928343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a17a20-2189-4716-bcb7-f959fae4e76b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:41.124931 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-76a17a20-2189-4716-bcb7-f959fae4e76b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b923435d156a4759a380d8800628ade2', 'request_id': 'req-76a17a20-2189-4716-bcb7-f959fae4e76b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:41.232530 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-76a17a20-2189-4716-bcb7-f959fae4e76b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b923435d156a4759a380d8800628ade2', 'request_id': 'req-76a17a20-2189-4716-bcb7-f959fae4e76b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:41.242667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-76a17a20-2189-4716-bcb7-f959fae4e76b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b923435d156a4759a380d8800628ade2', 'request_id': 'req-76a17a20-2189-4716-bcb7-f959fae4e76b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:41.254041 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-76a17a20-2189-4716-bcb7-f959fae4e76b admin admin] The token cache is being invalidate because project b923435d156a4759a380d8800628ade2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:41:41.254311 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-76a17a20-2189-4716-bcb7-f959fae4e76b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:41.255018 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-76a17a20-2189-4716-bcb7-f959fae4e76b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:41.257162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1303/2571] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:41:40 2022] DELETE /identity/v3/projects/b923435d156a4759a380d8800628ade2 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:41.262287 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6ad67-04ed-4c9c-b25e-2d2710e87a3b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:41.262780 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6ad67-04ed-4c9c-b25e-2d2710e87a3b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:41.262880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6ad67-04ed-4c9c-b25e-2d2710e87a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:41.281717 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-1bf6ad67-04ed-4c9c-b25e-2d2710e87a3b None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:41.282921 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-1bf6ad67-04ed-4c9c-b25e-2d2710e87a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:41.329272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:41.345398 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1269/2572] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:41.355635 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef3e55-05d2-4e23-ab93-156d3de2d616 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:41.355947 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef3e55-05d2-4e23-ab93-156d3de2d616 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:41.356225 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef3e55-05d2-4e23-ab93-156d3de2d616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:41.375085 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-94ef3e55-05d2-4e23-ab93-156d3de2d616 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:41.376263 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-94ef3e55-05d2-4e23-ab93-156d3de2d616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:41.385074 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1304/2573] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:41.389041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6ad67-04ed-4c9c-b25e-2d2710e87a3b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:41.399677 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6ad67-04ed-4c9c-b25e-2d2710e87a3b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:41.404358 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfdadae-d39f-45d2-b688-67f669923ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:41.405013 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdadae-d39f-45d2-b688-67f669923ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:41.405295 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdadae-d39f-45d2-b688-67f669923ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:41.405570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdadae-d39f-45d2-b688-67f669923ca1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:41.406288 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfdadae-d39f-45d2-b688-67f669923ca1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:41:41.406594 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfdadae-d39f-45d2-b688-67f669923ca1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:41.408224 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfdadae-d39f-45d2-b688-67f669923ca1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:41.420908 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1270/2574] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:41:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:41:41.425143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef3e55-05d2-4e23-ab93-156d3de2d616 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:41.436318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef3e55-05d2-4e23-ab93-156d3de2d616 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:41.441667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fe0ee-7484-410c-8569-ec5a57d4c09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:41.442370 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fe0ee-7484-410c-8569-ec5a57d4c09a admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:41.442686 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fe0ee-7484-410c-8569-ec5a57d4c09a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:41.442968 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fe0ee-7484-410c-8569-ec5a57d4c09a admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:41.443753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844fe0ee-7484-410c-8569-ec5a57d4c09a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:41.445674 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844fe0ee-7484-410c-8569-ec5a57d4c09a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:41.565458 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1305/2575] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:41:41 2022] POST /identity/v3/projects => generated 375 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:41:41.570321 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfdadae-d39f-45d2-b688-67f669923ca1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:41.580814 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfdadae-d39f-45d2-b688-67f669923ca1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:41.641638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d97774-7f95-4c3f-9525-9aaf7de618a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:41.642370 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d97774-7f95-4c3f-9525-9aaf7de618a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:41.642760 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d97774-7f95-4c3f-9525-9aaf7de618a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:41.643248 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d97774-7f95-4c3f-9525-9aaf7de618a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:41.644211 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d97774-7f95-4c3f-9525-9aaf7de618a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:41.646865 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d97774-7f95-4c3f-9525-9aaf7de618a5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:41.764492 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1271/2576] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:41:41 2022] POST /identity/v3/users => generated 288 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:41:41.768898 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fe0ee-7484-410c-8569-ec5a57d4c09a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:41.780441 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fe0ee-7484-410c-8569-ec5a57d4c09a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:41.784980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1ce65-e471-45bf-a0c9-9e943c5de01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:41.785628 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1ce65-e471-45bf-a0c9-9e943c5de01d admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:41.785908 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1ce65-e471-45bf-a0c9-9e943c5de01d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:41.786182 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1ce65-e471-45bf-a0c9-9e943c5de01d admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:41.786922 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a1ce65-e471-45bf-a0c9-9e943c5de01d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:41.788974 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a1ce65-e471-45bf-a0c9-9e943c5de01d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:41.799605 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1306/2577] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:41:41.803860 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d97774-7f95-4c3f-9525-9aaf7de618a5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:41.814161 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d97774-7f95-4c3f-9525-9aaf7de618a5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:41.820192 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85a4bd-5302-4472-9fb2-f3ef7872c40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:41.820995 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85a4bd-5302-4472-9fb2-f3ef7872c40a admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:41.821399 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85a4bd-5302-4472-9fb2-f3ef7872c40a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:41.821774 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85a4bd-5302-4472-9fb2-f3ef7872c40a admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:41.822593 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85a4bd-5302-4472-9fb2-f3ef7872c40a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:41.824450 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85a4bd-5302-4472-9fb2-f3ef7872c40a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:41.836125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1272/2578] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:41:41.840392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1ce65-e471-45bf-a0c9-9e943c5de01d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:41.851599 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1ce65-e471-45bf-a0c9-9e943c5de01d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:41.856339 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f49c8f-9d12-4f7b-ad4d-cd2bc4152d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:41.857217 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f49c8f-9d12-4f7b-ad4d-cd2bc4152d76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:41.860577 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f49c8f-9d12-4f7b-ad4d-cd2bc4152d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:41.860577 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f49c8f-9d12-4f7b-ad4d-cd2bc4152d76 admin admin] PATH_INFO: `/v3/projects/d5b5b9b012654bebb6a959e184f3f9fb/users/2691f4b7d4d446b7a3d1be014c478591/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:41.871428 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:41.874572 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f49c8f-9d12-4f7b-ad4d-cd2bc4152d76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5b5b9b012654bebb6a959e184f3f9fb, user_id=2691f4b7d4d446b7a3d1be014c478591, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:41.877274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f49c8f-9d12-4f7b-ad4d-cd2bc4152d76 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:41.975382 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1307/2579] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:41:41 2022] PUT /identity/v3/projects/d5b5b9b012654bebb6a959e184f3f9fb/users/2691f4b7d4d446b7a3d1be014c478591/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:41.985743 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b677b7c-05bd-42bb-8fab-bf89a0ebde9b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:41.986027 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b677b7c-05bd-42bb-8fab-bf89a0ebde9b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:41.986307 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b677b7c-05bd-42bb-8fab-bf89a0ebde9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:42.016142 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-3b677b7c-05bd-42bb-8fab-bf89a0ebde9b None None] MFA Rules not processed for user `2691f4b7d4d446b7a3d1be014c478591`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:42.017350 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3b677b7c-05bd-42bb-8fab-bf89a0ebde9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:42.088332 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1273/2580] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:42.095889 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f49c8f-9d12-4f7b-ad4d-cd2bc4152d76 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:42.097623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-09f49c8f-9d12-4f7b-ad4d-cd2bc4152d76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:42.158171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f49c8f-9d12-4f7b-ad4d-cd2bc4152d76 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:42.161849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba2b6e-d4dc-4e84-beb8-9b1b20a9efdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:42.162605 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba2b6e-d4dc-4e84-beb8-9b1b20a9efdf service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:42.162889 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba2b6e-d4dc-4e84-beb8-9b1b20a9efdf service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:42.163166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ba2b6e-d4dc-4e84-beb8-9b1b20a9efdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:42.171936 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ba2b6e-d4dc-4e84-beb8-9b1b20a9efdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:42.173460 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ba2b6e-d4dc-4e84-beb8-9b1b20a9efdf service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:42.183422 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1308/2581] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:41:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:41:44.672699 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b677b7c-05bd-42bb-8fab-bf89a0ebde9b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:44.674919 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3b677b7c-05bd-42bb-8fab-bf89a0ebde9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:44.724099 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:44.734959 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b677b7c-05bd-42bb-8fab-bf89a0ebde9b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:44.738558 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34953282-c5a4-4fc2-857d-17cb9da1f569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:44.739203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34953282-c5a4-4fc2-857d-17cb9da1f569 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:44.739518 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34953282-c5a4-4fc2-857d-17cb9da1f569 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:44.739797 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34953282-c5a4-4fc2-857d-17cb9da1f569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:44.741798 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-34953282-c5a4-4fc2-857d-17cb9da1f569 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:44.806379 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34953282-c5a4-4fc2-857d-17cb9da1f569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:44.807862 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34953282-c5a4-4fc2-857d-17cb9da1f569 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:44.818914 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1274/2582] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:41:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:41:46.254923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba2b6e-d4dc-4e84-beb8-9b1b20a9efdf service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:46.256445 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d9ba2b6e-d4dc-4e84-beb8-9b1b20a9efdf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:46.332210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ba2b6e-d4dc-4e84-beb8-9b1b20a9efdf service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:46.337468 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c232e2f-7961-4f49-b78c-8fb0d5f9a4e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a932a9390fdd47f3adcf053a60250944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:46.338133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c232e2f-7961-4f49-b78c-8fb0d5f9a4e3 service placement] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:46.338421 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c232e2f-7961-4f49-b78c-8fb0d5f9a4e3 service placement] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:46.338702 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c232e2f-7961-4f49-b78c-8fb0d5f9a4e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:46.340598 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2c232e2f-7961-4f49-b78c-8fb0d5f9a4e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:46.353846 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c232e2f-7961-4f49-b78c-8fb0d5f9a4e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:46.355526 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c232e2f-7961-4f49-b78c-8fb0d5f9a4e3 service placement] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:46.365962 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1309/2583] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:41:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:41:46.805551 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34953282-c5a4-4fc2-857d-17cb9da1f569 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:46.816437 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34953282-c5a4-4fc2-857d-17cb9da1f569 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:46.826467 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d945b-1b0b-42cd-9ee0-44dcc65d0f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:46.827371 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d945b-1b0b-42cd-9ee0-44dcc65d0f91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:46.827688 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d945b-1b0b-42cd-9ee0-44dcc65d0f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:46.828282 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d945b-1b0b-42cd-9ee0-44dcc65d0f91 admin admin] PATH_INFO: `/v3/users/2691f4b7d4d446b7a3d1be014c478591` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:46.829644 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212d945b-1b0b-42cd-9ee0-44dcc65d0f91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2691f4b7d4d446b7a3d1be014c478591)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:46.831871 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212d945b-1b0b-42cd-9ee0-44dcc65d0f91 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:47.018811 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c232e2f-7961-4f49-b78c-8fb0d5f9a4e3 service placement] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:47.020307 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2c232e2f-7961-4f49-b78c-8fb0d5f9a4e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:47.035428 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c232e2f-7961-4f49-b78c-8fb0d5f9a4e3 service placement] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:47.040961 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbc39e-47e3-488b-94b9-16a504410347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:47.042846 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbc39e-47e3-488b-94b9-16a504410347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:47.042846 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbc39e-47e3-488b-94b9-16a504410347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:47.042846 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbc39e-47e3-488b-94b9-16a504410347 admin admin] PATH_INFO: `/v3/users/3cead99b97164aa988c61998897f947d` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:47.043822 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbc39e-47e3-488b-94b9-16a504410347 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cead99b97164aa988c61998897f947d)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:47.045736 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbc39e-47e3-488b-94b9-16a504410347 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:47.105786 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-212d945b-1b0b-42cd-9ee0-44dcc65d0f91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2691f4b7d4d446b7a3d1be014c478591'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:47.106792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-212d945b-1b0b-42cd-9ee0-44dcc65d0f91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2691f4b7d4d446b7a3d1be014c478591'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:47.215188 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-212d945b-1b0b-42cd-9ee0-44dcc65d0f91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2691f4b7d4d446b7a3d1be014c478591'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:47.223744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-212d945b-1b0b-42cd-9ee0-44dcc65d0f91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2691f4b7d4d446b7a3d1be014c478591'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:47.224504 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-212d945b-1b0b-42cd-9ee0-44dcc65d0f91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2691f4b7d4d446b7a3d1be014c478591'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:47.246608 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1275/2584] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:41:46 2022] DELETE /identity/v3/users/2691f4b7d4d446b7a3d1be014c478591 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:47.251741 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc31f8-6dbf-4817-91a9-9185b3eeefa4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:47.252214 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc31f8-6dbf-4817-91a9-9185b3eeefa4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:47.252546 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc31f8-6dbf-4817-91a9-9185b3eeefa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:47.255815 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-14bbc39e-47e3-488b-94b9-16a504410347 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cead99b97164aa988c61998897f947d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:47.257060 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-14bbc39e-47e3-488b-94b9-16a504410347 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cead99b97164aa988c61998897f947d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:47.268451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-14bbc39e-47e3-488b-94b9-16a504410347 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cead99b97164aa988c61998897f947d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:47.275617 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-65bc31f8-6dbf-4817-91a9-9185b3eeefa4 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:47.276832 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-65bc31f8-6dbf-4817-91a9-9185b3eeefa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:47.344451 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1276/2585] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:47.352187 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc31f8-6dbf-4817-91a9-9185b3eeefa4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:47.353788 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-65bc31f8-6dbf-4817-91a9-9185b3eeefa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:47.363638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:47.417615 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc31f8-6dbf-4817-91a9-9185b3eeefa4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:47.421301 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03dd07f-7504-4c8f-a296-f54623f7672c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:47.421958 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03dd07f-7504-4c8f-a296-f54623f7672c service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:47.422238 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03dd07f-7504-4c8f-a296-f54623f7672c service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:47.422510 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03dd07f-7504-4c8f-a296-f54623f7672c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:47.433687 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03dd07f-7504-4c8f-a296-f54623f7672c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:47.435173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03dd07f-7504-4c8f-a296-f54623f7672c service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:47.444161 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-14bbc39e-47e3-488b-94b9-16a504410347 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cead99b97164aa988c61998897f947d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:47.445489 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-14bbc39e-47e3-488b-94b9-16a504410347 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cead99b97164aa988c61998897f947d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:47.466875 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:47.471336 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1310/2586] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:41:47 2022] DELETE /identity/v3/users/3cead99b97164aa988c61998897f947d => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:41:47.475913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38290f6d-6b27-4bf9-a173-1c25ca65832b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:47.476275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1277/2587] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:41:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:41:47.476823 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38290f6d-6b27-4bf9-a173-1c25ca65832b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:47.477267 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38290f6d-6b27-4bf9-a173-1c25ca65832b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:47.501373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-38290f6d-6b27-4bf9-a173-1c25ca65832b None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:47.503066 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-38290f6d-6b27-4bf9-a173-1c25ca65832b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:47.586618 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1311/2588] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:47.593362 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03dd07f-7504-4c8f-a296-f54623f7672c service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:47.594976 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a03dd07f-7504-4c8f-a296-f54623f7672c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:47.607436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03dd07f-7504-4c8f-a296-f54623f7672c service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:47.660200 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a732d1-7c1f-4467-a77d-a32463d9251f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:47.660865 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a732d1-7c1f-4467-a77d-a32463d9251f service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:47.661136 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a732d1-7c1f-4467-a77d-a32463d9251f service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:47.661468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a732d1-7c1f-4467-a77d-a32463d9251f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:47.671768 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a732d1-7c1f-4467-a77d-a32463d9251f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:47.673254 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a732d1-7c1f-4467-a77d-a32463d9251f service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:47.684218 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1278/2589] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:41:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:41:48.001099 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38290f6d-6b27-4bf9-a173-1c25ca65832b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:48.001968 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-38290f6d-6b27-4bf9-a173-1c25ca65832b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:48.029372 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38290f6d-6b27-4bf9-a173-1c25ca65832b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:48.036895 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8bd398-e076-4fc6-982d-b350c3507351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:48.039133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bd398-e076-4fc6-982d-b350c3507351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:48.039133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bd398-e076-4fc6-982d-b350c3507351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:48.039133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bd398-e076-4fc6-982d-b350c3507351 admin admin] PATH_INFO: `/v3/projects/d5b5b9b012654bebb6a959e184f3f9fb` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:48.041067 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8bd398-e076-4fc6-982d-b350c3507351 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5b5b9b012654bebb6a959e184f3f9fb)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:48.043020 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8bd398-e076-4fc6-982d-b350c3507351 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:48.220577 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ff8bd398-e076-4fc6-982d-b350c3507351 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5b5b9b012654bebb6a959e184f3f9fb', 'request_id': 'req-ff8bd398-e076-4fc6-982d-b350c3507351'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:48.246190 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a732d1-7c1f-4467-a77d-a32463d9251f service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:48.248387 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-03a732d1-7c1f-4467-a77d-a32463d9251f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:48.278832 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ff8bd398-e076-4fc6-982d-b350c3507351 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5b5b9b012654bebb6a959e184f3f9fb', 'request_id': 'req-ff8bd398-e076-4fc6-982d-b350c3507351'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:48.289637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ff8bd398-e076-4fc6-982d-b350c3507351 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5b5b9b012654bebb6a959e184f3f9fb', 'request_id': 'req-ff8bd398-e076-4fc6-982d-b350c3507351'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:48.289787 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a732d1-7c1f-4467-a77d-a32463d9251f service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:48.305836 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ff8bd398-e076-4fc6-982d-b350c3507351 admin admin] The token cache is being invalidate because project d5b5b9b012654bebb6a959e184f3f9fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:41:48.306197 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ff8bd398-e076-4fc6-982d-b350c3507351 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:48.306948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ff8bd398-e076-4fc6-982d-b350c3507351 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:48.308748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1312/2590] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:41:47 2022] DELETE /identity/v3/projects/d5b5b9b012654bebb6a959e184f3f9fb => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:48.313473 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31ce49-0808-490b-86ad-0c0c34d92fca None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:48.313830 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31ce49-0808-490b-86ad-0c0c34d92fca None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:48.314207 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31ce49-0808-490b-86ad-0c0c34d92fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:48.347586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-cd31ce49-0808-490b-86ad-0c0c34d92fca None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:48.349248 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cd31ce49-0808-490b-86ad-0c0c34d92fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:48.397594 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60c6cf-d926-488c-aff1-a78fb7963911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:48.399704 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60c6cf-d926-488c-aff1-a78fb7963911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:48.400073 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60c6cf-d926-488c-aff1-a78fb7963911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:48.400415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60c6cf-d926-488c-aff1-a78fb7963911 admin admin] PATH_INFO: `/v3/projects/2a02aa0af8ea43debd298f9b3dc98c3d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:48.402374 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60c6cf-d926-488c-aff1-a78fb7963911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a02aa0af8ea43debd298f9b3dc98c3d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:48.405315 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60c6cf-d926-488c-aff1-a78fb7963911 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:48.438154 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1313/2591] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:41:48.449236 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d07f6f-c30d-4bc7-8dad-d871d2447022 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:48.449528 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d07f6f-c30d-4bc7-8dad-d871d2447022 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:48.449891 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d07f6f-c30d-4bc7-8dad-d871d2447022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:48.474887 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-88d07f6f-c30d-4bc7-8dad-d871d2447022 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:48.479427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-88d07f6f-c30d-4bc7-8dad-d871d2447022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:48.486981 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1314/2592] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:48.491200 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d07f6f-c30d-4bc7-8dad-d871d2447022 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:48.513930 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d07f6f-c30d-4bc7-8dad-d871d2447022 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:48.522032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe60c6cf-d926-488c-aff1-a78fb7963911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a02aa0af8ea43debd298f9b3dc98c3d', 'request_id': 'req-fe60c6cf-d926-488c-aff1-a78fb7963911'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:48.530473 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:48.537085 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe60c6cf-d926-488c-aff1-a78fb7963911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a02aa0af8ea43debd298f9b3dc98c3d', 'request_id': 'req-fe60c6cf-d926-488c-aff1-a78fb7963911'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:48.544116 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe60c6cf-d926-488c-aff1-a78fb7963911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a02aa0af8ea43debd298f9b3dc98c3d', 'request_id': 'req-fe60c6cf-d926-488c-aff1-a78fb7963911'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:48.545259 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:48.623281 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f5c2e-6f4b-4175-88f8-838ae775bd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:48.624085 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f5c2e-6f4b-4175-88f8-838ae775bd11 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:48.624602 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f5c2e-6f4b-4175-88f8-838ae775bd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:48.625307 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f5c2e-6f4b-4175-88f8-838ae775bd11 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:48.625835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f5c2e-6f4b-4175-88f8-838ae775bd11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:41:48.626656 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f5c2e-6f4b-4175-88f8-838ae775bd11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:48.628357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f5c2e-6f4b-4175-88f8-838ae775bd11 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:48.637186 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe60c6cf-d926-488c-aff1-a78fb7963911 admin admin] The token cache is being invalidate because project 2a02aa0af8ea43debd298f9b3dc98c3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:41:48.638926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe60c6cf-d926-488c-aff1-a78fb7963911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:48.641910 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fe60c6cf-d926-488c-aff1-a78fb7963911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:48.643590 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1315/2593] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:41:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:41:48.644620 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1279/2594] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:41:48 2022] DELETE /identity/v3/projects/2a02aa0af8ea43debd298f9b3dc98c3d => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:41:48.649384 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f5c2e-6f4b-4175-88f8-838ae775bd11 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:48.651577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60c6cf-d926-488c-aff1-a78fb7963911 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:48.652395 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ef9f5c2e-6f4b-4175-88f8-838ae775bd11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:48.653031 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fe60c6cf-d926-488c-aff1-a78fb7963911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:48.666112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f5c2e-6f4b-4175-88f8-838ae775bd11 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:48.666847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60c6cf-d926-488c-aff1-a78fb7963911 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:48.670819 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f85787-c182-4def-85eb-289316086e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:48.671732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f85787-c182-4def-85eb-289316086e72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:48.672055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f85787-c182-4def-85eb-289316086e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:48.672332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f85787-c182-4def-85eb-289316086e72 admin admin] PATH_INFO: `/v3/users/3f2cadc3a8af42848a18a46e2bd9183f` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:48.672678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7994b-24b9-401c-937a-9343ac5bcbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:48.673427 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7994b-24b9-401c-937a-9343ac5bcbc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:48.673535 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f85787-c182-4def-85eb-289316086e72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f2cadc3a8af42848a18a46e2bd9183f)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:48.673778 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7994b-24b9-401c-937a-9343ac5bcbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:48.674631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7994b-24b9-401c-937a-9343ac5bcbc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:48.675419 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b7994b-24b9-401c-937a-9343ac5bcbc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:48.675567 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f85787-c182-4def-85eb-289316086e72 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:48.677542 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b7994b-24b9-401c-937a-9343ac5bcbc3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:48.807777 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1280/2595] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:41:48 2022] POST /identity/v3/projects => generated 369 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:41:48.815036 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7994b-24b9-401c-937a-9343ac5bcbc3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:48.876707 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7994b-24b9-401c-937a-9343ac5bcbc3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:48.881862 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968638db-d221-4aeb-bdd9-e422ceeb1504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:48.883355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968638db-d221-4aeb-bdd9-e422ceeb1504 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:48.883912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968638db-d221-4aeb-bdd9-e422ceeb1504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:48.884022 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968638db-d221-4aeb-bdd9-e422ceeb1504 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:48.884649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968638db-d221-4aeb-bdd9-e422ceeb1504 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:48.886709 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968638db-d221-4aeb-bdd9-e422ceeb1504 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:48.939608 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f6f85787-c182-4def-85eb-289316086e72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f2cadc3a8af42848a18a46e2bd9183f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:48.940291 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f6f85787-c182-4def-85eb-289316086e72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f2cadc3a8af42848a18a46e2bd9183f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:48.953782 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f6f85787-c182-4def-85eb-289316086e72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f2cadc3a8af42848a18a46e2bd9183f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:49.069537 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1281/2596] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:41:48 2022] POST /identity/v3/users => generated 285 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:41:49.073175 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968638db-d221-4aeb-bdd9-e422ceeb1504 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:49.084348 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968638db-d221-4aeb-bdd9-e422ceeb1504 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:49.089316 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb94b5-a40d-4d02-98bd-fd0c2db87858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:49.089995 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb94b5-a40d-4d02-98bd-fd0c2db87858 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:49.090282 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb94b5-a40d-4d02-98bd-fd0c2db87858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:49.090560 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb94b5-a40d-4d02-98bd-fd0c2db87858 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:49.091322 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb94b5-a40d-4d02-98bd-fd0c2db87858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:49.093168 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb94b5-a40d-4d02-98bd-fd0c2db87858 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:49.096168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f6f85787-c182-4def-85eb-289316086e72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f2cadc3a8af42848a18a46e2bd9183f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:49.096993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-f6f85787-c182-4def-85eb-289316086e72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f2cadc3a8af42848a18a46e2bd9183f'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:49.108141 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1282/2597] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:41:49.119669 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb94b5-a40d-4d02-98bd-fd0c2db87858 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:49.121631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b1fb94b5-a40d-4d02-98bd-fd0c2db87858 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:49.124465 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1316/2598] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:41:48 2022] DELETE /identity/v3/users/3f2cadc3a8af42848a18a46e2bd9183f => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:41:49.146620 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb94b5-a40d-4d02-98bd-fd0c2db87858 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:49.162101 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f85787-c182-4def-85eb-289316086e72 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:49.164691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f6f85787-c182-4def-85eb-289316086e72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:49.238408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a08d9e-9c05-4266-a8bc-fd1ec066bbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:49.238676 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a08d9e-9c05-4266-a8bc-fd1ec066bbd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:49.238980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a08d9e-9c05-4266-a8bc-fd1ec066bbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:49.239257 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a08d9e-9c05-4266-a8bc-fd1ec066bbd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:49.240034 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a08d9e-9c05-4266-a8bc-fd1ec066bbd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:49.241675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a08d9e-9c05-4266-a8bc-fd1ec066bbd1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:49.253259 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1283/2599] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:49 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:41:49.258160 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a08d9e-9c05-4266-a8bc-fd1ec066bbd1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:49.269207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a08d9e-9c05-4266-a8bc-fd1ec066bbd1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:49.277271 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369711b-3949-40a7-a4bd-ba599527325f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:49.278212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369711b-3949-40a7-a4bd-ba599527325f admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:49.278496 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369711b-3949-40a7-a4bd-ba599527325f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:49.278787 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369711b-3949-40a7-a4bd-ba599527325f admin admin] PATH_INFO: `/v3/projects/41a84a287aad4e089bf30d9c8fb91986/users/528fc09c1cde4b50ab91560be809d9a6/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:49.281683 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f85787-c182-4def-85eb-289316086e72 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:49.286498 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817cdd7-ca2f-4e5a-819b-3d1dc37619f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:49.287408 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817cdd7-ca2f-4e5a-819b-3d1dc37619f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:49.288108 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817cdd7-ca2f-4e5a-819b-3d1dc37619f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:49.288731 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817cdd7-ca2f-4e5a-819b-3d1dc37619f7 admin admin] PATH_INFO: `/v3/users/7261ab46436e4546b8331aa8387cfa1c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:49.290291 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817cdd7-ca2f-4e5a-819b-3d1dc37619f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7261ab46436e4546b8331aa8387cfa1c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:49.292482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817cdd7-ca2f-4e5a-819b-3d1dc37619f7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:49.302480 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369711b-3949-40a7-a4bd-ba599527325f admin admin] RBAC: Authorizing `identity:create_grant(project_id=41a84a287aad4e089bf30d9c8fb91986, user_id=528fc09c1cde4b50ab91560be809d9a6, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:49.305335 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369711b-3949-40a7-a4bd-ba599527325f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:49.387831 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1284/2600] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:41:49 2022] PUT /identity/v3/projects/41a84a287aad4e089bf30d9c8fb91986/users/528fc09c1cde4b50ab91560be809d9a6/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:49.410860 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad433c3-5bfe-4c8a-bd39-b1de81ad9406 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:49.411161 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad433c3-5bfe-4c8a-bd39-b1de81ad9406 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:49.411451 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad433c3-5bfe-4c8a-bd39-b1de81ad9406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:49.442561 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4ad433c3-5bfe-4c8a-bd39-b1de81ad9406 None None] MFA Rules not processed for user `528fc09c1cde4b50ab91560be809d9a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:49.443828 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4ad433c3-5bfe-4c8a-bd39-b1de81ad9406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:49.514334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1285/2601] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:49 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:41:49.527310 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e817cdd7-ca2f-4e5a-819b-3d1dc37619f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7261ab46436e4546b8331aa8387cfa1c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:49.527665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad433c3-5bfe-4c8a-bd39-b1de81ad9406 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:49.528078 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e817cdd7-ca2f-4e5a-819b-3d1dc37619f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7261ab46436e4546b8331aa8387cfa1c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:49.529231 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4ad433c3-5bfe-4c8a-bd39-b1de81ad9406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:49.535607 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e817cdd7-ca2f-4e5a-819b-3d1dc37619f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7261ab46436e4546b8331aa8387cfa1c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:49.586601 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad433c3-5bfe-4c8a-bd39-b1de81ad9406 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:49.590389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7f2ba-a632-4985-bded-dcb7b1d08051 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec41c84c9375437590c42621ca5b400a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:49.591067 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7f2ba-a632-4985-bded-dcb7b1d08051 service glance] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:49.591352 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7f2ba-a632-4985-bded-dcb7b1d08051 service glance] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:49.591682 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7f2ba-a632-4985-bded-dcb7b1d08051 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:49.600304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7f2ba-a632-4985-bded-dcb7b1d08051 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:49.602051 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f7f2ba-a632-4985-bded-dcb7b1d08051 service glance] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:49.603702 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e817cdd7-ca2f-4e5a-819b-3d1dc37619f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7261ab46436e4546b8331aa8387cfa1c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:49.604344 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-e817cdd7-ca2f-4e5a-819b-3d1dc37619f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7261ab46436e4546b8331aa8387cfa1c'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:49.613766 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1286/2602] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:41:49 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:41:49.614649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7f2ba-a632-4985-bded-dcb7b1d08051 service glance] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:49.616139 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-65f7f2ba-a632-4985-bded-dcb7b1d08051 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:49.662980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1317/2603] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:41:49 2022] DELETE /identity/v3/users/7261ab46436e4546b8331aa8387cfa1c => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:41:49.703585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7f2ba-a632-4985-bded-dcb7b1d08051 service glance] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:49.765851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:DELETE=3,keystone:SELECT=91,keystone:INSERT=6 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:49.776293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb8551b-76ec-4af3-9d45-638d537adc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:49.777171 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8551b-76ec-4af3-9d45-638d537adc93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:49.777452 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8551b-76ec-4af3-9d45-638d537adc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:49.777809 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8551b-76ec-4af3-9d45-638d537adc93 admin admin] PATH_INFO: `/v3/projects/f96bc9295bd64603a6e4a41fc3fe4670` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:49.778907 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb8551b-76ec-4af3-9d45-638d537adc93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f96bc9295bd64603a6e4a41fc3fe4670)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:49.780843 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb8551b-76ec-4af3-9d45-638d537adc93 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:49.907781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5cb8551b-76ec-4af3-9d45-638d537adc93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f96bc9295bd64603a6e4a41fc3fe4670', 'request_id': 'req-5cb8551b-76ec-4af3-9d45-638d537adc93'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:49.915601 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5cb8551b-76ec-4af3-9d45-638d537adc93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f96bc9295bd64603a6e4a41fc3fe4670', 'request_id': 'req-5cb8551b-76ec-4af3-9d45-638d537adc93'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:49.921657 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5cb8551b-76ec-4af3-9d45-638d537adc93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f96bc9295bd64603a6e4a41fc3fe4670', 'request_id': 'req-5cb8551b-76ec-4af3-9d45-638d537adc93'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:50.038265 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817cdd7-ca2f-4e5a-819b-3d1dc37619f7 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:50.040383 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e817cdd7-ca2f-4e5a-819b-3d1dc37619f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:50.057049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817cdd7-ca2f-4e5a-819b-3d1dc37619f7 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:50.113538 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:50.119069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5cb8551b-76ec-4af3-9d45-638d537adc93 admin admin] The token cache is being invalidate because project f96bc9295bd64603a6e4a41fc3fe4670 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:41:50.119386 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5cb8551b-76ec-4af3-9d45-638d537adc93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:50.120435 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5cb8551b-76ec-4af3-9d45-638d537adc93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:50.122887 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1287/2604] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:41:49 2022] DELETE /identity/v3/projects/f96bc9295bd64603a6e4a41fc3fe4670 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:50.127488 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f00566-0df1-4f0a-b623-8d0968168498 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:50.128160 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f00566-0df1-4f0a-b623-8d0968168498 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:50.128438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f00566-0df1-4f0a-b623-8d0968168498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:50.129950 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb858f4d-ec94-457f-8c09-cf44bf31b454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:50.130819 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb858f4d-ec94-457f-8c09-cf44bf31b454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:50.131099 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb858f4d-ec94-457f-8c09-cf44bf31b454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:50.131380 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb858f4d-ec94-457f-8c09-cf44bf31b454 admin admin] PATH_INFO: `/v3/projects/b66c389f8c5a47b2b97819e449c64ba0` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:50.133260 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb858f4d-ec94-457f-8c09-cf44bf31b454 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b66c389f8c5a47b2b97819e449c64ba0)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:50.135184 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb858f4d-ec94-457f-8c09-cf44bf31b454 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:50.148326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d5f00566-0df1-4f0a-b623-8d0968168498 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:50.149558 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d5f00566-0df1-4f0a-b623-8d0968168498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:50.157735 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1288/2605] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:41:50.166100 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3557f75-3d7c-48d6-9fe9-5e7475a85125 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:50.166391 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3557f75-3d7c-48d6-9fe9-5e7475a85125 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:50.166668 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3557f75-3d7c-48d6-9fe9-5e7475a85125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:50.185700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f3557f75-3d7c-48d6-9fe9-5e7475a85125 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:50.186999 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f3557f75-3d7c-48d6-9fe9-5e7475a85125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:50.195304 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1289/2606] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:41:50.199551 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3557f75-3d7c-48d6-9fe9-5e7475a85125 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:50.208674 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3557f75-3d7c-48d6-9fe9-5e7475a85125 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:50.213424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf4fee-319f-4021-a39f-621afc7df56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:50.214077 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf4fee-319f-4021-a39f-621afc7df56b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:50.214362 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf4fee-319f-4021-a39f-621afc7df56b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:50.214682 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf4fee-319f-4021-a39f-621afc7df56b admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:50.215399 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf4fee-319f-4021-a39f-621afc7df56b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:41:50.215738 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf4fee-319f-4021-a39f-621afc7df56b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:50.217323 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf4fee-319f-4021-a39f-621afc7df56b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:50.230235 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1290/2607] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:41:50.235009 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf4fee-319f-4021-a39f-621afc7df56b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:50.249668 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf4fee-319f-4021-a39f-621afc7df56b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:50.260595 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-eb858f4d-ec94-457f-8c09-cf44bf31b454 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b66c389f8c5a47b2b97819e449c64ba0', 'request_id': 'req-eb858f4d-ec94-457f-8c09-cf44bf31b454'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:50.309239 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9123d244-8d6e-45e4-86b2-7dbd80b8ff59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:50.309909 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9123d244-8d6e-45e4-86b2-7dbd80b8ff59 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:50.310237 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9123d244-8d6e-45e4-86b2-7dbd80b8ff59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:50.310515 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9123d244-8d6e-45e4-86b2-7dbd80b8ff59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:50.311200 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9123d244-8d6e-45e4-86b2-7dbd80b8ff59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:50.313148 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9123d244-8d6e-45e4-86b2-7dbd80b8ff59 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:50.352662 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-eb858f4d-ec94-457f-8c09-cf44bf31b454 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b66c389f8c5a47b2b97819e449c64ba0', 'request_id': 'req-eb858f4d-ec94-457f-8c09-cf44bf31b454'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:50.360010 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-eb858f4d-ec94-457f-8c09-cf44bf31b454 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b66c389f8c5a47b2b97819e449c64ba0', 'request_id': 'req-eb858f4d-ec94-457f-8c09-cf44bf31b454'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:50.370875 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-eb858f4d-ec94-457f-8c09-cf44bf31b454 admin admin] The token cache is being invalidate because project b66c389f8c5a47b2b97819e449c64ba0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:41:50.371173 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-eb858f4d-ec94-457f-8c09-cf44bf31b454 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:50.371861 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-eb858f4d-ec94-457f-8c09-cf44bf31b454 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:50.373593 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1318/2608] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:41:50 2022] DELETE /identity/v3/projects/b66c389f8c5a47b2b97819e449c64ba0 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:41:50.378563 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a0780-2b21-460d-bc67-d2f995151506 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:50.378870 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a0780-2b21-460d-bc67-d2f995151506 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:50.379171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a0780-2b21-460d-bc67-d2f995151506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:50.398567 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-f25a0780-2b21-460d-bc67-d2f995151506 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:50.400031 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-f25a0780-2b21-460d-bc67-d2f995151506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:50.408118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1319/2609] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:41:50.416485 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2dc096-3d95-40ca-a7eb-57cfe0e93de8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:50.422440 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2dc096-3d95-40ca-a7eb-57cfe0e93de8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:50.422440 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2dc096-3d95-40ca-a7eb-57cfe0e93de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:50.422675 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1291/2610] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:41:50 2022] POST /identity/v3/projects => generated 393 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:41:50.428111 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9123d244-8d6e-45e4-86b2-7dbd80b8ff59 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:50.431371 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9123d244-8d6e-45e4-86b2-7dbd80b8ff59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:50.436243 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8f2dc096-3d95-40ca-a7eb-57cfe0e93de8 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:50.437482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8f2dc096-3d95-40ca-a7eb-57cfe0e93de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:50.500438 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1320/2611] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:50.500828 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9123d244-8d6e-45e4-86b2-7dbd80b8ff59 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:50.504961 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2dc096-3d95-40ca-a7eb-57cfe0e93de8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:50.505650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c319d-ecc7-480e-a95a-6a4d585ad54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:50.506323 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c319d-ecc7-480e-a95a-6a4d585ad54f admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:50.506600 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c319d-ecc7-480e-a95a-6a4d585ad54f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:50.506875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c319d-ecc7-480e-a95a-6a4d585ad54f admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:50.507740 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97c319d-ecc7-480e-a95a-6a4d585ad54f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:50.509647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97c319d-ecc7-480e-a95a-6a4d585ad54f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:50.514457 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2dc096-3d95-40ca-a7eb-57cfe0e93de8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:50.519204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36b3dd-d94b-4906-966a-21d0d81f9446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:50.519912 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36b3dd-d94b-4906-966a-21d0d81f9446 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:50.520202 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36b3dd-d94b-4906-966a-21d0d81f9446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:50.520482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36b3dd-d94b-4906-966a-21d0d81f9446 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:50.521206 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36b3dd-d94b-4906-966a-21d0d81f9446 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:41:50.521519 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36b3dd-d94b-4906-966a-21d0d81f9446 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:50.523933 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36b3dd-d94b-4906-966a-21d0d81f9446 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:50.536307 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1321/2612] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:41:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:41:50.541542 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36b3dd-d94b-4906-966a-21d0d81f9446 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:50.552291 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36b3dd-d94b-4906-966a-21d0d81f9446 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:50.557289 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6336f2-eac4-47db-9096-1e03c7826f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:50.557979 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6336f2-eac4-47db-9096-1e03c7826f8e admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:50.558302 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6336f2-eac4-47db-9096-1e03c7826f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:50.558590 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6336f2-eac4-47db-9096-1e03c7826f8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:50.565920 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6336f2-eac4-47db-9096-1e03c7826f8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:50.565920 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6336f2-eac4-47db-9096-1e03c7826f8e admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:50.596716 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1292/2613] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:41:50 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:41:50.597912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c319d-ecc7-480e-a95a-6a4d585ad54f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:50.609428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97c319d-ecc7-480e-a95a-6a4d585ad54f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:50.614157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0701ef-1a7d-4cf3-ad9f-4f2403aedb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:50.614820 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0701ef-1a7d-4cf3-ad9f-4f2403aedb74 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:50.615206 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0701ef-1a7d-4cf3-ad9f-4f2403aedb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:50.615365 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0701ef-1a7d-4cf3-ad9f-4f2403aedb74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:50.616189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0701ef-1a7d-4cf3-ad9f-4f2403aedb74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:50.617920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0701ef-1a7d-4cf3-ad9f-4f2403aedb74 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:50.628633 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1293/2614] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:41:50.633794 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0701ef-1a7d-4cf3-ad9f-4f2403aedb74 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:50.645221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0701ef-1a7d-4cf3-ad9f-4f2403aedb74 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:50.657870 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac4baa-1cf4-4efd-a658-c229423a05cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:50.657870 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac4baa-1cf4-4efd-a658-c229423a05cf admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:50.657870 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac4baa-1cf4-4efd-a658-c229423a05cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:50.657870 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac4baa-1cf4-4efd-a658-c229423a05cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:50.658757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac4baa-1cf4-4efd-a658-c229423a05cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:50.658757 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac4baa-1cf4-4efd-a658-c229423a05cf admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:50.669267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1294/2615] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:41:50.673725 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac4baa-1cf4-4efd-a658-c229423a05cf admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:50.685340 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac4baa-1cf4-4efd-a658-c229423a05cf admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:50.691008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e1c0a-b979-4251-8805-f946bc2f8afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:50.692317 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e1c0a-b979-4251-8805-f946bc2f8afd admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:50.692604 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e1c0a-b979-4251-8805-f946bc2f8afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:50.692910 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e1c0a-b979-4251-8805-f946bc2f8afd admin admin] PATH_INFO: `/v3/projects/98ade04633d642c9902c7b1bf90895ad/users/9bb30c860eb84e3da5baec134ce66002/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:50.711813 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e1c0a-b979-4251-8805-f946bc2f8afd admin admin] RBAC: Authorizing `identity:create_grant(project_id=98ade04633d642c9902c7b1bf90895ad, user_id=9bb30c860eb84e3da5baec134ce66002, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:50.715270 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e1c0a-b979-4251-8805-f946bc2f8afd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:50.834121 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1322/2616] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:41:50 2022] POST /identity/v3/projects => generated 403 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:41:50.838548 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6336f2-eac4-47db-9096-1e03c7826f8e admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:50.904160 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6336f2-eac4-47db-9096-1e03c7826f8e admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:50.908796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34552841-8eb5-4e23-b106-9110161096f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:50.909463 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34552841-8eb5-4e23-b106-9110161096f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:50.909749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34552841-8eb5-4e23-b106-9110161096f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:50.910025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34552841-8eb5-4e23-b106-9110161096f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:50.910713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34552841-8eb5-4e23-b106-9110161096f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:50.912658 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34552841-8eb5-4e23-b106-9110161096f4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:50.957934 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1295/2617] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:41:50 2022] PUT /identity/v3/projects/98ade04633d642c9902c7b1bf90895ad/users/9bb30c860eb84e3da5baec134ce66002/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:41:50.970333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd88a4-c7a5-4b89-a75b-02f0dfa6bd90 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:50.970609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd88a4-c7a5-4b89-a75b-02f0dfa6bd90 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:50.970886 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd88a4-c7a5-4b89-a75b-02f0dfa6bd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:51.000708 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-95dd88a4-c7a5-4b89-a75b-02f0dfa6bd90 None None] MFA Rules not processed for user `9bb30c860eb84e3da5baec134ce66002`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:51.001555 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-95dd88a4-c7a5-4b89-a75b-02f0dfa6bd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:51.068314 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1296/2618] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:51.073314 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd88a4-c7a5-4b89-a75b-02f0dfa6bd90 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:51.110986 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1323/2619] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:41:50 2022] POST /identity/v3/users => generated 302 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:41:51.117166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34552841-8eb5-4e23-b106-9110161096f4 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:51.128085 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34552841-8eb5-4e23-b106-9110161096f4 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:51.145480 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd88a4-c7a5-4b89-a75b-02f0dfa6bd90 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:51.150901 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc84d7a-14d6-4424-b60b-9bf95a56c900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:51.153249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc84d7a-14d6-4424-b60b-9bf95a56c900 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:51.153249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc84d7a-14d6-4424-b60b-9bf95a56c900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:51.153249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc84d7a-14d6-4424-b60b-9bf95a56c900 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:51.153249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc84d7a-14d6-4424-b60b-9bf95a56c900 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:51.154937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc84d7a-14d6-4424-b60b-9bf95a56c900 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:51.188413 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb84cd0-5c0c-45b1-96a5-b64b304f0402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:51.189088 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb84cd0-5c0c-45b1-96a5-b64b304f0402 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:51.189374 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb84cd0-5c0c-45b1-96a5-b64b304f0402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:51.189657 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb84cd0-5c0c-45b1-96a5-b64b304f0402 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:51.190391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb84cd0-5c0c-45b1-96a5-b64b304f0402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:51.192294 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb84cd0-5c0c-45b1-96a5-b64b304f0402 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:51.202995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1324/2620] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:41:51.206940 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb84cd0-5c0c-45b1-96a5-b64b304f0402 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:51.217080 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb84cd0-5c0c-45b1-96a5-b64b304f0402 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:51.221734 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204405f-fb26-4328-bb94-ee69ebc47f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:51.222429 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204405f-fb26-4328-bb94-ee69ebc47f97 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:51.222719 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204405f-fb26-4328-bb94-ee69ebc47f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:51.223002 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204405f-fb26-4328-bb94-ee69ebc47f97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:51.223749 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b204405f-fb26-4328-bb94-ee69ebc47f97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:51.225434 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b204405f-fb26-4328-bb94-ee69ebc47f97 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:51.235755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1325/2621] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:41:51.239917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204405f-fb26-4328-bb94-ee69ebc47f97 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:51.251338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204405f-fb26-4328-bb94-ee69ebc47f97 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:51.256348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe194a-af4f-40bf-a977-1e3c3ec98229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:51.257927 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe194a-af4f-40bf-a977-1e3c3ec98229 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:51.258688 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe194a-af4f-40bf-a977-1e3c3ec98229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:51.258688 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe194a-af4f-40bf-a977-1e3c3ec98229 admin admin] PATH_INFO: `/v3/projects/1555691c61a942fb8f20a671f1d638b3/users/7e9cf90add624f46908f4982ec908884/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:51.277368 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1297/2622] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:41:51 2022] POST /identity/v3/projects => generated 393 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:41:51.277594 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe194a-af4f-40bf-a977-1e3c3ec98229 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1555691c61a942fb8f20a671f1d638b3, user_id=7e9cf90add624f46908f4982ec908884, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:51.277781 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe194a-af4f-40bf-a977-1e3c3ec98229 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:51.278430 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc84d7a-14d6-4424-b60b-9bf95a56c900 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:51.318349 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:51.341684 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1326/2623] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:41:51 2022] PUT /identity/v3/projects/1555691c61a942fb8f20a671f1d638b3/users/7e9cf90add624f46908f4982ec908884/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:41:51.350045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc84d7a-14d6-4424-b60b-9bf95a56c900 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:51.351994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006cce8-12f8-4940-bc49-7324ca3a8961 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:51.352320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006cce8-12f8-4940-bc49-7324ca3a8961 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:51.352592 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006cce8-12f8-4940-bc49-7324ca3a8961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:51.381094 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-e006cce8-12f8-4940-bc49-7324ca3a8961 None None] MFA Rules not processed for user `7e9cf90add624f46908f4982ec908884`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:51.383517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-e006cce8-12f8-4940-bc49-7324ca3a8961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:51.417785 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02bc54-da8d-41e6-8740-766a3d17bfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:51.418836 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02bc54-da8d-41e6-8740-766a3d17bfd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:51.419141 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02bc54-da8d-41e6-8740-766a3d17bfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:51.419506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc02bc54-da8d-41e6-8740-766a3d17bfd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:51.420365 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc02bc54-da8d-41e6-8740-766a3d17bfd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:51.422441 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc02bc54-da8d-41e6-8740-766a3d17bfd0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:51.440023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:51.467014 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1327/2624] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:51 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:41:51.471865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006cce8-12f8-4940-bc49-7324ca3a8961 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:51.484667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006cce8-12f8-4940-bc49-7324ca3a8961 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:51.489524 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322abbb-36a7-4899-a6dc-d84585722d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:51.490196 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322abbb-36a7-4899-a6dc-d84585722d35 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:51.490481 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322abbb-36a7-4899-a6dc-d84585722d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:51.490763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322abbb-36a7-4899-a6dc-d84585722d35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:51.491715 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0322abbb-36a7-4899-a6dc-d84585722d35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:51.494510 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0322abbb-36a7-4899-a6dc-d84585722d35 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:51.536889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1298/2625] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:41:51 2022] POST /identity/v3/users => generated 303 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:41:51.541468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02bc54-da8d-41e6-8740-766a3d17bfd0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:51.552330 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc02bc54-da8d-41e6-8740-766a3d17bfd0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:51.558057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d87f73-862f-4874-a816-751569158af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:51.558732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d87f73-862f-4874-a816-751569158af7 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:51.559014 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d87f73-862f-4874-a816-751569158af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:51.559291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d87f73-862f-4874-a816-751569158af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:51.560094 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d87f73-862f-4874-a816-751569158af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:51.561926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d87f73-862f-4874-a816-751569158af7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:51.573079 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1299/2626] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:41:51.577097 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d87f73-862f-4874-a816-751569158af7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:51.587952 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d87f73-862f-4874-a816-751569158af7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:51.592927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9eed7-3066-46b4-b969-a9fcd18843a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:51.593863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9eed7-3066-46b4-b969-a9fcd18843a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:51.594148 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9eed7-3066-46b4-b969-a9fcd18843a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:51.594425 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d9eed7-3066-46b4-b969-a9fcd18843a4 admin admin] PATH_INFO: `/v3/projects/ee3b293b20ba47dfab442562c3c1c322/users/8c48007a5a6147e5a1992f70639481c6/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:51.610214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1328/2627] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:41:51 2022] POST /identity/v3/projects => generated 405 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:41:51.610939 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9eed7-3066-46b4-b969-a9fcd18843a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee3b293b20ba47dfab442562c3c1c322, user_id=8c48007a5a6147e5a1992f70639481c6, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:51.611073 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322abbb-36a7-4899-a6dc-d84585722d35 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:51.613635 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d9eed7-3066-46b4-b969-a9fcd18843a4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:51.642532 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:51.691976 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322abbb-36a7-4899-a6dc-d84585722d35 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:51.703178 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0ddf3-9415-4dd7-89a3-28f2831836f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:51.703178 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0ddf3-9415-4dd7-89a3-28f2831836f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:51.703178 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0ddf3-9415-4dd7-89a3-28f2831836f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:51.703178 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0ddf3-9415-4dd7-89a3-28f2831836f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:51.703724 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a0ddf3-9415-4dd7-89a3-28f2831836f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:51.703724 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a0ddf3-9415-4dd7-89a3-28f2831836f8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:51.729784 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1300/2628] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:41:51 2022] PUT /identity/v3/projects/ee3b293b20ba47dfab442562c3c1c322/users/8c48007a5a6147e5a1992f70639481c6/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:41:51.733834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9eed7-3066-46b4-b969-a9fcd18843a4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:51.798361 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d9eed7-3066-46b4-b969-a9fcd18843a4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:51.803876 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259844cf-dbee-4bd0-aaae-e8f55a20ba44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:51.805267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259844cf-dbee-4bd0-aaae-e8f55a20ba44 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:51.805457 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259844cf-dbee-4bd0-aaae-e8f55a20ba44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:51.805871 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259844cf-dbee-4bd0-aaae-e8f55a20ba44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:51.806669 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259844cf-dbee-4bd0-aaae-e8f55a20ba44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:51.808347 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259844cf-dbee-4bd0-aaae-e8f55a20ba44 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:51.819398 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1301/2629] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:41:51.823656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259844cf-dbee-4bd0-aaae-e8f55a20ba44 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:51.833652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259844cf-dbee-4bd0-aaae-e8f55a20ba44 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:51.838315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1329/2630] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:41:51 2022] POST /identity/v3/users => generated 309 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:41:51.842662 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0ddf3-9415-4dd7-89a3-28f2831836f8 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:51.842895 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe7f4d-905d-46be-a64e-9734731bc312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:51.843853 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe7f4d-905d-46be-a64e-9734731bc312 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:51.844131 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe7f4d-905d-46be-a64e-9734731bc312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:51.844403 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe7f4d-905d-46be-a64e-9734731bc312 admin admin] PATH_INFO: `/v3/projects/ee3b293b20ba47dfab442562c3c1c322/users/8c48007a5a6147e5a1992f70639481c6/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:51.846490 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fe7f4d-905d-46be-a64e-9734731bc312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee3b293b20ba47dfab442562c3c1c322, user_id=8c48007a5a6147e5a1992f70639481c6, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:51.849208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fe7f4d-905d-46be-a64e-9734731bc312 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:51.854252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0ddf3-9415-4dd7-89a3-28f2831836f8 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:51.863053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dae08b-db8c-4cca-b927-fd118c461898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:51.863053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dae08b-db8c-4cca-b927-fd118c461898 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:51.863053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dae08b-db8c-4cca-b927-fd118c461898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:51.863053 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dae08b-db8c-4cca-b927-fd118c461898 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:51.863516 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1302/2631] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:41:51 2022] PUT /identity/v3/projects/ee3b293b20ba47dfab442562c3c1c322/users/8c48007a5a6147e5a1992f70639481c6/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:41:51.863640 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dae08b-db8c-4cca-b927-fd118c461898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:51.864595 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dae08b-db8c-4cca-b927-fd118c461898 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:51.872173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026b3e1-6a18-47f7-8c36-ee649dd584f1 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:51.872582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026b3e1-6a18-47f7-8c36-ee649dd584f1 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:51.872709 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026b3e1-6a18-47f7-8c36-ee649dd584f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:51.878947 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1330/2632] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:41:51.880929 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dae08b-db8c-4cca-b927-fd118c461898 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:51.890717 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dae08b-db8c-4cca-b927-fd118c461898 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:51.901766 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f026b3e1-6a18-47f7-8c36-ee649dd584f1 None None] MFA Rules not processed for user `8c48007a5a6147e5a1992f70639481c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:51.902980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f026b3e1-6a18-47f7-8c36-ee649dd584f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:51.951073 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2aa1c6-d1df-432d-8729-1f18e94a5178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:51.952046 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2aa1c6-d1df-432d-8729-1f18e94a5178 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:51.952328 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2aa1c6-d1df-432d-8729-1f18e94a5178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:51.952903 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2aa1c6-d1df-432d-8729-1f18e94a5178 admin admin] PATH_INFO: `/v3/projects/e98f8697701d4282ad5b4a5e2d9d1f3d/users/7feee81d5ce24adcbc0d0d9cc7555cbe/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:51.974512 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2aa1c6-d1df-432d-8729-1f18e94a5178 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e98f8697701d4282ad5b4a5e2d9d1f3d, user_id=7feee81d5ce24adcbc0d0d9cc7555cbe, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:51.977569 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1303/2633] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:51 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:41:51.978249 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2aa1c6-d1df-432d-8729-1f18e94a5178 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:51.988464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026b3e1-6a18-47f7-8c36-ee649dd584f1 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:51.990011 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f026b3e1-6a18-47f7-8c36-ee649dd584f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:52.057084 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026b3e1-6a18-47f7-8c36-ee649dd584f1 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:52.060960 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f7775-e8aa-4cda-a634-544adb2727df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:52.061606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f7775-e8aa-4cda-a634-544adb2727df service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:52.061887 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f7775-e8aa-4cda-a634-544adb2727df service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:52.062163 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f7775-e8aa-4cda-a634-544adb2727df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:52.073363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f7775-e8aa-4cda-a634-544adb2727df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:52.074868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f7775-e8aa-4cda-a634-544adb2727df service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:52.082842 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1331/2634] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:41:51 2022] PUT /identity/v3/projects/e98f8697701d4282ad5b4a5e2d9d1f3d/users/7feee81d5ce24adcbc0d0d9cc7555cbe/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:52.086015 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1304/2635] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:41:51 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:41:52.087806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2aa1c6-d1df-432d-8729-1f18e94a5178 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:52.105381 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:52.138460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f7775-e8aa-4cda-a634-544adb2727df service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:52.138460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-798f7775-e8aa-4cda-a634-544adb2727df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:52.149926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f7775-e8aa-4cda-a634-544adb2727df service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:52.158401 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2aa1c6-d1df-432d-8729-1f18e94a5178 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:52.164359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:52.167117 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ef381-dcb0-4a27-bccf-fdae04df8191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:52.167117 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ef381-dcb0-4a27-bccf-fdae04df8191 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:52.167117 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ef381-dcb0-4a27-bccf-fdae04df8191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:52.167117 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ef381-dcb0-4a27-bccf-fdae04df8191 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:52.167670 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ef381-dcb0-4a27-bccf-fdae04df8191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:52.168389 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ef381-dcb0-4a27-bccf-fdae04df8191 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:52.183136 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1332/2636] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:41:52.184557 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ef381-dcb0-4a27-bccf-fdae04df8191 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:52.196587 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ef381-dcb0-4a27-bccf-fdae04df8191 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:52.201340 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1094d7f-e82c-41e1-8871-07e927095f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:52.202252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1094d7f-e82c-41e1-8871-07e927095f93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:52.202470 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739f24f-158a-4e10-9ed2-54ad8890e0d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:52.202604 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1094d7f-e82c-41e1-8871-07e927095f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:52.202806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1094d7f-e82c-41e1-8871-07e927095f93 admin admin] PATH_INFO: `/v3/projects/e98f8697701d4282ad5b4a5e2d9d1f3d/users/7feee81d5ce24adcbc0d0d9cc7555cbe/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:52.203170 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739f24f-158a-4e10-9ed2-54ad8890e0d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:52.203459 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739f24f-158a-4e10-9ed2-54ad8890e0d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:52.203807 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1739f24f-158a-4e10-9ed2-54ad8890e0d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:52.204867 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1094d7f-e82c-41e1-8871-07e927095f93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e98f8697701d4282ad5b4a5e2d9d1f3d, user_id=7feee81d5ce24adcbc0d0d9cc7555cbe, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:52.207704 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1094d7f-e82c-41e1-8871-07e927095f93 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:52.220087 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1333/2637] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:41:52 2022] PUT /identity/v3/projects/e98f8697701d4282ad5b4a5e2d9d1f3d/users/7feee81d5ce24adcbc0d0d9cc7555cbe/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:41:52.231897 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294525b-59c7-4b66-8ce0-e94d4b111001 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:52.232239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294525b-59c7-4b66-8ce0-e94d4b111001 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:52.232514 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294525b-59c7-4b66-8ce0-e94d4b111001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:52.249125 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:52.262296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8294525b-59c7-4b66-8ce0-e94d4b111001 None None] MFA Rules not processed for user `7feee81d5ce24adcbc0d0d9cc7555cbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:52.263586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8294525b-59c7-4b66-8ce0-e94d4b111001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:52.265905 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1739f24f-158a-4e10-9ed2-54ad8890e0d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:52.267356 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1739f24f-158a-4e10-9ed2-54ad8890e0d2 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:52.277669 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1305/2638] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:41:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:41:52.306691 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:52.340263 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1334/2639] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:52 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:41:52.354968 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739f24f-158a-4e10-9ed2-54ad8890e0d2 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:52.364101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1739f24f-158a-4e10-9ed2-54ad8890e0d2 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:52.417863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:52.422511 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562b38b-eba9-4897-a626-9ac1aaf208aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:52.423246 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562b38b-eba9-4897-a626-9ac1aaf208aa service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:52.423652 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562b38b-eba9-4897-a626-9ac1aaf208aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:52.423928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562b38b-eba9-4897-a626-9ac1aaf208aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:52.433620 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562b38b-eba9-4897-a626-9ac1aaf208aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:52.435044 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562b38b-eba9-4897-a626-9ac1aaf208aa service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:52.445681 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1306/2640] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:41:52 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:41:52.593641 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294525b-59c7-4b66-8ce0-e94d4b111001 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:52.612479 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:52.668249 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294525b-59c7-4b66-8ce0-e94d4b111001 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:52.672967 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e9e1d-bae2-4e4a-9680-6e4986432f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:52.673846 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e9e1d-bae2-4e4a-9680-6e4986432f30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:52.674126 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e9e1d-bae2-4e4a-9680-6e4986432f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:52.674400 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e9e1d-bae2-4e4a-9680-6e4986432f30 admin admin] PATH_INFO: `/v3/users/7e9cf90add624f46908f4982ec908884` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:52.675680 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e9e1d-bae2-4e4a-9680-6e4986432f30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e9cf90add624f46908f4982ec908884)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:52.678102 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e9e1d-bae2-4e4a-9680-6e4986432f30 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:53.034991 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b03e9e1d-bae2-4e4a-9680-6e4986432f30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e9cf90add624f46908f4982ec908884'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:53.035714 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b03e9e1d-bae2-4e4a-9680-6e4986432f30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e9cf90add624f46908f4982ec908884'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:53.049713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b03e9e1d-bae2-4e4a-9680-6e4986432f30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e9cf90add624f46908f4982ec908884'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:53.150899 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b03e9e1d-bae2-4e4a-9680-6e4986432f30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e9cf90add624f46908f4982ec908884'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:53.151637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b03e9e1d-bae2-4e4a-9680-6e4986432f30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e9cf90add624f46908f4982ec908884'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:53.171758 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1335/2641] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:41:52 2022] DELETE /identity/v3/users/7e9cf90add624f46908f4982ec908884 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:53.176594 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608905e6-4677-4c68-a019-31d5b760ed93 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:53.176868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608905e6-4677-4c68-a019-31d5b760ed93 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:53.177153 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608905e6-4677-4c68-a019-31d5b760ed93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:53.196008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-608905e6-4677-4c68-a019-31d5b760ed93 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:53.197164 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-608905e6-4677-4c68-a019-31d5b760ed93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:53.269332 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1307/2642] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:53.277175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e9e1d-bae2-4e4a-9680-6e4986432f30 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:53.278614 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b03e9e1d-bae2-4e4a-9680-6e4986432f30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:53.343107 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e9e1d-bae2-4e4a-9680-6e4986432f30 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:53.347071 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653ff56-367c-468b-8064-dfeb41efdac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:53.347766 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653ff56-367c-468b-8064-dfeb41efdac1 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:53.348146 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653ff56-367c-468b-8064-dfeb41efdac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:53.348511 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8653ff56-367c-468b-8064-dfeb41efdac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:53.360934 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653ff56-367c-468b-8064-dfeb41efdac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:53.362408 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8653ff56-367c-468b-8064-dfeb41efdac1 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:53.373517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1336/2643] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:41:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:41:53.817280 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608905e6-4677-4c68-a019-31d5b760ed93 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:53.817776 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-608905e6-4677-4c68-a019-31d5b760ed93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:53.829930 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608905e6-4677-4c68-a019-31d5b760ed93 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:53.834744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a820f7b-8d96-436d-a416-216fb0ee9327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:53.835930 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a820f7b-8d96-436d-a416-216fb0ee9327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:53.840414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a820f7b-8d96-436d-a416-216fb0ee9327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:53.840414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a820f7b-8d96-436d-a416-216fb0ee9327 admin admin] PATH_INFO: `/v3/projects/1555691c61a942fb8f20a671f1d638b3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:53.840414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a820f7b-8d96-436d-a416-216fb0ee9327 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1555691c61a942fb8f20a671f1d638b3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:53.840414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a820f7b-8d96-436d-a416-216fb0ee9327 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:53.959644 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2a820f7b-8d96-436d-a416-216fb0ee9327 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1555691c61a942fb8f20a671f1d638b3', 'request_id': 'req-2a820f7b-8d96-436d-a416-216fb0ee9327'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:53.967021 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2a820f7b-8d96-436d-a416-216fb0ee9327 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1555691c61a942fb8f20a671f1d638b3', 'request_id': 'req-2a820f7b-8d96-436d-a416-216fb0ee9327'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:53.973266 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2a820f7b-8d96-436d-a416-216fb0ee9327 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1555691c61a942fb8f20a671f1d638b3', 'request_id': 'req-2a820f7b-8d96-436d-a416-216fb0ee9327'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:53.974889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:54.090561 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2a820f7b-8d96-436d-a416-216fb0ee9327 admin admin] The token cache is being invalidate because project 1555691c61a942fb8f20a671f1d638b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:41:54.097212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2a820f7b-8d96-436d-a416-216fb0ee9327 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:54.097212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-2a820f7b-8d96-436d-a416-216fb0ee9327 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:54.097212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1308/2644] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:41:53 2022] DELETE /identity/v3/projects/1555691c61a942fb8f20a671f1d638b3 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:54.097532 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653ff56-367c-468b-8064-dfeb41efdac1 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:54.107979 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8653ff56-367c-468b-8064-dfeb41efdac1 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:54.146657 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:54.171828 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744e8bb-174e-411a-a239-db3ff5eb1ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:54.172633 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744e8bb-174e-411a-a239-db3ff5eb1ff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:54.173040 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744e8bb-174e-411a-a239-db3ff5eb1ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:54.173324 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744e8bb-174e-411a-a239-db3ff5eb1ff7 admin admin] PATH_INFO: `/v3/users/7feee81d5ce24adcbc0d0d9cc7555cbe` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:54.174640 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744e8bb-174e-411a-a239-db3ff5eb1ff7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7feee81d5ce24adcbc0d0d9cc7555cbe)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:54.176500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744e8bb-174e-411a-a239-db3ff5eb1ff7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:54.367422 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b744e8bb-174e-411a-a239-db3ff5eb1ff7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7feee81d5ce24adcbc0d0d9cc7555cbe'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:54.368073 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b744e8bb-174e-411a-a239-db3ff5eb1ff7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7feee81d5ce24adcbc0d0d9cc7555cbe'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:54.375153 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b744e8bb-174e-411a-a239-db3ff5eb1ff7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7feee81d5ce24adcbc0d0d9cc7555cbe'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:54.432620 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b744e8bb-174e-411a-a239-db3ff5eb1ff7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7feee81d5ce24adcbc0d0d9cc7555cbe'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:54.433203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b744e8bb-174e-411a-a239-db3ff5eb1ff7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7feee81d5ce24adcbc0d0d9cc7555cbe'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:54.452647 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1337/2645] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:41:54 2022] DELETE /identity/v3/users/7feee81d5ce24adcbc0d0d9cc7555cbe => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:54.875423 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a820f7b-8d96-436d-a416-216fb0ee9327 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:54.877328 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2a820f7b-8d96-436d-a416-216fb0ee9327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:54.899935 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a820f7b-8d96-436d-a416-216fb0ee9327 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:54.972188 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ef369d-3100-4b47-b1b3-18dc756e94e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:54.973068 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef369d-3100-4b47-b1b3-18dc756e94e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:54.973355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef369d-3100-4b47-b1b3-18dc756e94e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:54.973644 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ef369d-3100-4b47-b1b3-18dc756e94e1 admin admin] PATH_INFO: `/v3/projects/e98f8697701d4282ad5b4a5e2d9d1f3d` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:54.975006 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef369d-3100-4b47-b1b3-18dc756e94e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e98f8697701d4282ad5b4a5e2d9d1f3d)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:54.977245 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ef369d-3100-4b47-b1b3-18dc756e94e1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:55.096517 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c6ef369d-3100-4b47-b1b3-18dc756e94e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e98f8697701d4282ad5b4a5e2d9d1f3d', 'request_id': 'req-c6ef369d-3100-4b47-b1b3-18dc756e94e1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:55.103389 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c6ef369d-3100-4b47-b1b3-18dc756e94e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e98f8697701d4282ad5b4a5e2d9d1f3d', 'request_id': 'req-c6ef369d-3100-4b47-b1b3-18dc756e94e1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:55.108907 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c6ef369d-3100-4b47-b1b3-18dc756e94e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e98f8697701d4282ad5b4a5e2d9d1f3d', 'request_id': 'req-c6ef369d-3100-4b47-b1b3-18dc756e94e1'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:55.173012 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c6ef369d-3100-4b47-b1b3-18dc756e94e1 admin admin] The token cache is being invalidate because project e98f8697701d4282ad5b4a5e2d9d1f3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:41:55.173310 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c6ef369d-3100-4b47-b1b3-18dc756e94e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:55.173977 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-c6ef369d-3100-4b47-b1b3-18dc756e94e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:55.175658 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1309/2646] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:41:54 2022] DELETE /identity/v3/projects/e98f8697701d4282ad5b4a5e2d9d1f3d => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:55.181735 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ed46e-f47c-44ad-827c-fec25be7453b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:55.182579 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ed46e-f47c-44ad-827c-fec25be7453b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:55.182998 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ed46e-f47c-44ad-827c-fec25be7453b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:55.202713 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-be5ed46e-f47c-44ad-827c-fec25be7453b None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:55.203986 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-be5ed46e-f47c-44ad-827c-fec25be7453b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:55.266827 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1338/2647] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:55.274796 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5f35d-84b7-4343-9553-bf63d4f93d0a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:55.275331 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5f35d-84b7-4343-9553-bf63d4f93d0a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:55.275815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5f35d-84b7-4343-9553-bf63d4f93d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:55.297641 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-bca5f35d-84b7-4343-9553-bf63d4f93d0a None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:55.298894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-bca5f35d-84b7-4343-9553-bf63d4f93d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:55.308104 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1310/2648] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:41:55.312433 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ed46e-f47c-44ad-827c-fec25be7453b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:55.327612 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ed46e-f47c-44ad-827c-fec25be7453b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:55.332334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c39c6b-e063-490b-8bd5-c146d94b900b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:55.332994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c39c6b-e063-490b-8bd5-c146d94b900b admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:55.333277 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c39c6b-e063-490b-8bd5-c146d94b900b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:55.333630 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c39c6b-e063-490b-8bd5-c146d94b900b admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:55.334351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c39c6b-e063-490b-8bd5-c146d94b900b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:41:55.334658 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c39c6b-e063-490b-8bd5-c146d94b900b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:55.336990 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c39c6b-e063-490b-8bd5-c146d94b900b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:55.348797 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1339/2649] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:41:55.353990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5f35d-84b7-4343-9553-bf63d4f93d0a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:55.364252 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5f35d-84b7-4343-9553-bf63d4f93d0a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:55.370936 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e7366-a261-4a05-9826-853f839d23f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:55.371698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e7366-a261-4a05-9826-853f839d23f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:55.371988 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e7366-a261-4a05-9826-853f839d23f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:55.372260 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e7366-a261-4a05-9826-853f839d23f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:55.372956 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e7366-a261-4a05-9826-853f839d23f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:55.374870 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e7366-a261-4a05-9826-853f839d23f2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:55.451252 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1311/2650] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:41:55 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:41:55.454767 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c39c6b-e063-490b-8bd5-c146d94b900b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:55.466425 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c39c6b-e063-490b-8bd5-c146d94b900b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:55.511444 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:55.527587 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2caa4a-b9ff-42fa-ad95-e24967ba2910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:55.528264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2caa4a-b9ff-42fa-ad95-e24967ba2910 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:55.528550 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2caa4a-b9ff-42fa-ad95-e24967ba2910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:55.528830 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2caa4a-b9ff-42fa-ad95-e24967ba2910 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:55.529544 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2caa4a-b9ff-42fa-ad95-e24967ba2910 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:55.531483 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2caa4a-b9ff-42fa-ad95-e24967ba2910 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:55.638337 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1340/2651] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:41:55 2022] POST /identity/v3/users => generated 294 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:41:55.642289 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e7366-a261-4a05-9826-853f839d23f2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:55.654994 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e7366-a261-4a05-9826-853f839d23f2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:55.660611 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a4ed0-63c8-4749-9f06-4eb3b08f243b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:55.661305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a4ed0-63c8-4749-9f06-4eb3b08f243b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:55.661650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a4ed0-63c8-4749-9f06-4eb3b08f243b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:55.661951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a4ed0-63c8-4749-9f06-4eb3b08f243b admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:55.662683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a4ed0-63c8-4749-9f06-4eb3b08f243b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:55.665279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a4ed0-63c8-4749-9f06-4eb3b08f243b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:55.676578 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1312/2652] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:41:55.681265 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2caa4a-b9ff-42fa-ad95-e24967ba2910 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:55.691920 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2caa4a-b9ff-42fa-ad95-e24967ba2910 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:55.697747 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b796e42-1d25-4c1f-bc7b-48530f46c157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:55.698475 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b796e42-1d25-4c1f-bc7b-48530f46c157 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:55.698794 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b796e42-1d25-4c1f-bc7b-48530f46c157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:55.699078 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b796e42-1d25-4c1f-bc7b-48530f46c157 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:55.699861 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b796e42-1d25-4c1f-bc7b-48530f46c157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:55.701547 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b796e42-1d25-4c1f-bc7b-48530f46c157 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:55.713217 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1341/2653] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:41:55.717544 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a4ed0-63c8-4749-9f06-4eb3b08f243b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:55.728770 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a4ed0-63c8-4749-9f06-4eb3b08f243b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:55.733394 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbd49a-3131-4497-8751-497627f66cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:55.734265 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbd49a-3131-4497-8751-497627f66cb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:55.734548 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbd49a-3131-4497-8751-497627f66cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:55.734825 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbd49a-3131-4497-8751-497627f66cb6 admin admin] PATH_INFO: `/v3/projects/b1d795ffa7f44314b3a74ec658117645/users/c628648fa7fd4d13a96235051c87cfbb/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:55.750814 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dbd49a-3131-4497-8751-497627f66cb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1d795ffa7f44314b3a74ec658117645, user_id=c628648fa7fd4d13a96235051c87cfbb, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:55.753611 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dbd49a-3131-4497-8751-497627f66cb6 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:55.861771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1313/2654] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:41:55 2022] PUT /identity/v3/projects/b1d795ffa7f44314b3a74ec658117645/users/c628648fa7fd4d13a96235051c87cfbb/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:41:55.876447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8367b8-b6ff-4649-96b7-1478bee8c635 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:55.876861 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8367b8-b6ff-4649-96b7-1478bee8c635 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:55.877352 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8367b8-b6ff-4649-96b7-1478bee8c635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:55.905023 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fb8367b8-b6ff-4649-96b7-1478bee8c635 None None] MFA Rules not processed for user `c628648fa7fd4d13a96235051c87cfbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:55.906241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fb8367b8-b6ff-4649-96b7-1478bee8c635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:55.974732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1342/2655] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:55.979309 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbd49a-3131-4497-8751-497627f66cb6 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:56.000062 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:56.046055 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbd49a-3131-4497-8751-497627f66cb6 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:56.054966 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2319d36-18ce-4664-b625-c12ced57c459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:56.055731 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2319d36-18ce-4664-b625-c12ced57c459 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:56.056368 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2319d36-18ce-4664-b625-c12ced57c459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:56.056509 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2319d36-18ce-4664-b625-c12ced57c459 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:56.057192 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2319d36-18ce-4664-b625-c12ced57c459 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:56.059339 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2319d36-18ce-4664-b625-c12ced57c459 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:56.142577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1314/2656] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:41:55 2022] POST /identity/v3/projects => generated 387 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:41:56.147069 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8367b8-b6ff-4649-96b7-1478bee8c635 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:56.160069 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8367b8-b6ff-4649-96b7-1478bee8c635 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:56.222165 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37457c-4410-42ab-8bc4-804b716c1850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:56.222917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37457c-4410-42ab-8bc4-804b716c1850 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:56.223142 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37457c-4410-42ab-8bc4-804b716c1850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:56.223509 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37457c-4410-42ab-8bc4-804b716c1850 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:56.224220 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37457c-4410-42ab-8bc4-804b716c1850 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:56.226974 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37457c-4410-42ab-8bc4-804b716c1850 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:56.346754 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1343/2657] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:41:56 2022] POST /identity/v3/users => generated 294 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:41:56.350740 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2319d36-18ce-4664-b625-c12ced57c459 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:56.362343 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2319d36-18ce-4664-b625-c12ced57c459 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:56.367656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610da1d-52cd-4e39-9640-e2df003c3072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:56.368348 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610da1d-52cd-4e39-9640-e2df003c3072 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:56.368649 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610da1d-52cd-4e39-9640-e2df003c3072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:56.368927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d610da1d-52cd-4e39-9640-e2df003c3072 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:56.369657 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d610da1d-52cd-4e39-9640-e2df003c3072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:56.371355 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d610da1d-52cd-4e39-9640-e2df003c3072 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:56.382775 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1315/2658] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:41:56.386862 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37457c-4410-42ab-8bc4-804b716c1850 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:56.397275 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37457c-4410-42ab-8bc4-804b716c1850 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:56.402561 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16105f54-46f8-43b0-b1ef-856684d5f6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:56.403219 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16105f54-46f8-43b0-b1ef-856684d5f6f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:56.403564 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16105f54-46f8-43b0-b1ef-856684d5f6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:56.403816 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16105f54-46f8-43b0-b1ef-856684d5f6f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:56.404586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16105f54-46f8-43b0-b1ef-856684d5f6f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:56.406307 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16105f54-46f8-43b0-b1ef-856684d5f6f4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:56.418778 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1344/2659] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:41:56.423004 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610da1d-52cd-4e39-9640-e2df003c3072 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:56.433322 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d610da1d-52cd-4e39-9640-e2df003c3072 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:56.438171 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875d648-ba06-4e3e-a359-d52c92922929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:56.439062 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875d648-ba06-4e3e-a359-d52c92922929 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:56.439347 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875d648-ba06-4e3e-a359-d52c92922929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:56.439717 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875d648-ba06-4e3e-a359-d52c92922929 admin admin] PATH_INFO: `/v3/projects/66af0479fdf84b8bae47494cdef54f53/users/ac82df04bc7949b5a519a38e3d37285d/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:56.456541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875d648-ba06-4e3e-a359-d52c92922929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66af0479fdf84b8bae47494cdef54f53, user_id=ac82df04bc7949b5a519a38e3d37285d, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:56.459264 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875d648-ba06-4e3e-a359-d52c92922929 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:56.535567 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1316/2660] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:41:56 2022] PUT /identity/v3/projects/66af0479fdf84b8bae47494cdef54f53/users/ac82df04bc7949b5a519a38e3d37285d/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:41:56.546340 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86ee3f-8f8c-4cf6-9ded-71e023e04cb0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:56.546632 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86ee3f-8f8c-4cf6-9ded-71e023e04cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:56.546914 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86ee3f-8f8c-4cf6-9ded-71e023e04cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:56.577775 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1b86ee3f-8f8c-4cf6-9ded-71e023e04cb0 None None] MFA Rules not processed for user `ac82df04bc7949b5a519a38e3d37285d`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:56.579085 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1b86ee3f-8f8c-4cf6-9ded-71e023e04cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:56.599527 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:56.653730 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1345/2661] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:56.654232 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875d648-ba06-4e3e-a359-d52c92922929 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:56.723238 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875d648-ba06-4e3e-a359-d52c92922929 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:56.728483 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6be7f0-db6f-4ace-8ed7-e536e607f5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:56.729151 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6be7f0-db6f-4ace-8ed7-e536e607f5a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:56.729458 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6be7f0-db6f-4ace-8ed7-e536e607f5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:56.729769 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6be7f0-db6f-4ace-8ed7-e536e607f5a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:56.730459 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6be7f0-db6f-4ace-8ed7-e536e607f5a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:56.732432 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6be7f0-db6f-4ace-8ed7-e536e607f5a8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:56.865469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1317/2662] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:41:56 2022] POST /identity/v3/projects => generated 387 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:41:56.869088 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86ee3f-8f8c-4cf6-9ded-71e023e04cb0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:56.881361 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86ee3f-8f8c-4cf6-9ded-71e023e04cb0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:56.959609 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff322e3-13f1-49ae-ada6-bcc8b56bf1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:56.961194 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff322e3-13f1-49ae-ada6-bcc8b56bf1bc admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:56.961482 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff322e3-13f1-49ae-ada6-bcc8b56bf1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:56.961779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff322e3-13f1-49ae-ada6-bcc8b56bf1bc admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:56.962528 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff322e3-13f1-49ae-ada6-bcc8b56bf1bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:56.964745 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff322e3-13f1-49ae-ada6-bcc8b56bf1bc admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:57.038696 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1346/2663] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:41:56 2022] POST /identity/v3/users => generated 300 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:41:57.043164 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6be7f0-db6f-4ace-8ed7-e536e607f5a8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:57.055770 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6be7f0-db6f-4ace-8ed7-e536e607f5a8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:57.060655 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d5daa-6130-4220-aaba-0b080cc00dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:57.061326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d5daa-6130-4220-aaba-0b080cc00dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:57.061696 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d5daa-6130-4220-aaba-0b080cc00dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:57.061894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d5daa-6130-4220-aaba-0b080cc00dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:57.062775 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d5daa-6130-4220-aaba-0b080cc00dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:57.064264 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d5daa-6130-4220-aaba-0b080cc00dd1 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:57.081130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1318/2664] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:41:57.086189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff322e3-13f1-49ae-ada6-bcc8b56bf1bc admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:57.099885 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff322e3-13f1-49ae-ada6-bcc8b56bf1bc admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:57.107911 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949d1e4-5e26-40f4-9deb-03a2b5d42c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:57.107911 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949d1e4-5e26-40f4-9deb-03a2b5d42c87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:57.107911 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949d1e4-5e26-40f4-9deb-03a2b5d42c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:57.107911 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a949d1e4-5e26-40f4-9deb-03a2b5d42c87 admin admin] PATH_INFO: `/v3/projects/5d1099f456ea4f369f34257aed5009e1/users/1b649c84b1a744ba89af7f2b03a8917e/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:57.124079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a949d1e4-5e26-40f4-9deb-03a2b5d42c87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d1099f456ea4f369f34257aed5009e1, user_id=1b649c84b1a744ba89af7f2b03a8917e, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:57.126830 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a949d1e4-5e26-40f4-9deb-03a2b5d42c87 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:57.200169 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1347/2665] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:41:57 2022] PUT /identity/v3/projects/5d1099f456ea4f369f34257aed5009e1/users/1b649c84b1a744ba89af7f2b03a8917e/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:57.204520 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d5daa-6130-4220-aaba-0b080cc00dd1 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:57.218889 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d5daa-6130-4220-aaba-0b080cc00dd1 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:57.250013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949d1e4-5e26-40f4-9deb-03a2b5d42c87 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:57.252116 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a949d1e4-5e26-40f4-9deb-03a2b5d42c87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:57.267710 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:57.287989 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b510142-d0a8-4ae5-b434-10d28eccee50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:57.288744 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b510142-d0a8-4ae5-b434-10d28eccee50 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:57.288926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b510142-d0a8-4ae5-b434-10d28eccee50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:57.289205 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b510142-d0a8-4ae5-b434-10d28eccee50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:57.289938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b510142-d0a8-4ae5-b434-10d28eccee50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:57.291939 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b510142-d0a8-4ae5-b434-10d28eccee50 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:57.306288 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1319/2666] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:41:57.310472 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b510142-d0a8-4ae5-b434-10d28eccee50 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:57.321546 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b510142-d0a8-4ae5-b434-10d28eccee50 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:57.326489 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe1d38-8e87-45d2-b4c7-94b0177a2bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:57.327740 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe1d38-8e87-45d2-b4c7-94b0177a2bb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:57.327988 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe1d38-8e87-45d2-b4c7-94b0177a2bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:57.328224 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe1d38-8e87-45d2-b4c7-94b0177a2bb4 admin admin] PATH_INFO: `/v3/projects/5d1099f456ea4f369f34257aed5009e1/users/1b649c84b1a744ba89af7f2b03a8917e/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:57.330173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe1d38-8e87-45d2-b4c7-94b0177a2bb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d1099f456ea4f369f34257aed5009e1, user_id=1b649c84b1a744ba89af7f2b03a8917e, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:57.333011 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe1d38-8e87-45d2-b4c7-94b0177a2bb4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:57.335263 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a949d1e4-5e26-40f4-9deb-03a2b5d42c87 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:57.342473 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2bedb-0177-47e5-8389-fba49a1be10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:57.343999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2bedb-0177-47e5-8389-fba49a1be10a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:57.344494 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2bedb-0177-47e5-8389-fba49a1be10a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:57.344828 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce2bedb-0177-47e5-8389-fba49a1be10a admin admin] PATH_INFO: `/v3/users/9bb30c860eb84e3da5baec134ce66002` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:57.345630 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1320/2667] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:41:57 2022] PUT /identity/v3/projects/5d1099f456ea4f369f34257aed5009e1/users/1b649c84b1a744ba89af7f2b03a8917e/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:41:57.346076 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce2bedb-0177-47e5-8389-fba49a1be10a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bb30c860eb84e3da5baec134ce66002)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:57.348304 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce2bedb-0177-47e5-8389-fba49a1be10a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:57.357583 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996ab2e-23ed-41fc-b3ae-a883d423beed None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:57.357876 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996ab2e-23ed-41fc-b3ae-a883d423beed None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:57.358159 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d996ab2e-23ed-41fc-b3ae-a883d423beed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:57.396650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-d996ab2e-23ed-41fc-b3ae-a883d423beed None None] MFA Rules not processed for user `1b649c84b1a744ba89af7f2b03a8917e`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:57.397804 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d996ab2e-23ed-41fc-b3ae-a883d423beed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:57.468087 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:57.478815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1321/2668] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:57 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:57.493545 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996ab2e-23ed-41fc-b3ae-a883d423beed None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:57.495164 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-d996ab2e-23ed-41fc-b3ae-a883d423beed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:57.564918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d996ab2e-23ed-41fc-b3ae-a883d423beed None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:57.569195 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be14231-cd9c-454a-b06c-36b3cb8b2f6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:57.570273 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be14231-cd9c-454a-b06c-36b3cb8b2f6b service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:57.570273 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be14231-cd9c-454a-b06c-36b3cb8b2f6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:57.570638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be14231-cd9c-454a-b06c-36b3cb8b2f6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:57.574554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3ce2bedb-0177-47e5-8389-fba49a1be10a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bb30c860eb84e3da5baec134ce66002'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:57.575271 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3ce2bedb-0177-47e5-8389-fba49a1be10a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bb30c860eb84e3da5baec134ce66002'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:57.583575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be14231-cd9c-454a-b06c-36b3cb8b2f6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:57.583847 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3ce2bedb-0177-47e5-8389-fba49a1be10a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bb30c860eb84e3da5baec134ce66002'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:57.585237 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be14231-cd9c-454a-b06c-36b3cb8b2f6b service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:57.596780 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1322/2669] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:41:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:41:57.660307 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3ce2bedb-0177-47e5-8389-fba49a1be10a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bb30c860eb84e3da5baec134ce66002'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:57.660935 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3ce2bedb-0177-47e5-8389-fba49a1be10a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bb30c860eb84e3da5baec134ce66002'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:57.687284 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1348/2670] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:41:57 2022] DELETE /identity/v3/users/9bb30c860eb84e3da5baec134ce66002 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:41:57.688640 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067cc7c-4715-4b17-a32b-e8a710c05e6b None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:57.688938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067cc7c-4715-4b17-a32b-e8a710c05e6b None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:57.689270 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067cc7c-4715-4b17-a32b-e8a710c05e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:57.707838 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-9067cc7c-4715-4b17-a32b-e8a710c05e6b None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:57.709376 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9067cc7c-4715-4b17-a32b-e8a710c05e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:57.761912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:57.774436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1323/2671] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:57.786622 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2bedb-0177-47e5-8389-fba49a1be10a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:57.788209 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3ce2bedb-0177-47e5-8389-fba49a1be10a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:57.854765 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce2bedb-0177-47e5-8389-fba49a1be10a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:57.858680 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5816d38-61f9-47a5-8a66-5bb2d4a6da6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:57.859444 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5816d38-61f9-47a5-8a66-5bb2d4a6da6e service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:57.859766 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5816d38-61f9-47a5-8a66-5bb2d4a6da6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:57.860049 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5816d38-61f9-47a5-8a66-5bb2d4a6da6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:57.872070 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5816d38-61f9-47a5-8a66-5bb2d4a6da6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:57.873541 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5816d38-61f9-47a5-8a66-5bb2d4a6da6e service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:57.885918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1349/2672] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:41:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:41:58.356713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067cc7c-4715-4b17-a32b-e8a710c05e6b None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:58.358226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-9067cc7c-4715-4b17-a32b-e8a710c05e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:58.370580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067cc7c-4715-4b17-a32b-e8a710c05e6b None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:58.375273 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a41b4e-8a35-40ad-8c96-2c0d10c63018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:58.376187 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a41b4e-8a35-40ad-8c96-2c0d10c63018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:58.376465 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a41b4e-8a35-40ad-8c96-2c0d10c63018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:58.376754 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a41b4e-8a35-40ad-8c96-2c0d10c63018 admin admin] PATH_INFO: `/v3/projects/98ade04633d642c9902c7b1bf90895ad` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:58.377927 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a41b4e-8a35-40ad-8c96-2c0d10c63018 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98ade04633d642c9902c7b1bf90895ad)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:58.379911 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a41b4e-8a35-40ad-8c96-2c0d10c63018 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:58.501469 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e7a41b4e-8a35-40ad-8c96-2c0d10c63018 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98ade04633d642c9902c7b1bf90895ad', 'request_id': 'req-e7a41b4e-8a35-40ad-8c96-2c0d10c63018'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:58.509990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e7a41b4e-8a35-40ad-8c96-2c0d10c63018 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98ade04633d642c9902c7b1bf90895ad', 'request_id': 'req-e7a41b4e-8a35-40ad-8c96-2c0d10c63018'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:58.522619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e7a41b4e-8a35-40ad-8c96-2c0d10c63018 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98ade04633d642c9902c7b1bf90895ad', 'request_id': 'req-e7a41b4e-8a35-40ad-8c96-2c0d10c63018'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:58.603808 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e7a41b4e-8a35-40ad-8c96-2c0d10c63018 admin admin] The token cache is being invalidate because project 98ade04633d642c9902c7b1bf90895ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:41:58.604130 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e7a41b4e-8a35-40ad-8c96-2c0d10c63018 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:58.604982 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e7a41b4e-8a35-40ad-8c96-2c0d10c63018 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:58.606666 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1324/2673] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:41:58 2022] DELETE /identity/v3/projects/98ade04633d642c9902c7b1bf90895ad => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:58.611217 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5816d38-61f9-47a5-8a66-5bb2d4a6da6e service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:58.622838 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5816d38-61f9-47a5-8a66-5bb2d4a6da6e service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:58.696831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47037314-f1fe-4705-9776-018c73c31ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:58.697722 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47037314-f1fe-4705-9776-018c73c31ffa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:58.697999 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47037314-f1fe-4705-9776-018c73c31ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:58.698272 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47037314-f1fe-4705-9776-018c73c31ffa admin admin] PATH_INFO: `/v3/users/8c48007a5a6147e5a1992f70639481c6` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:58.699488 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47037314-f1fe-4705-9776-018c73c31ffa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c48007a5a6147e5a1992f70639481c6)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:58.701505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47037314-f1fe-4705-9776-018c73c31ffa admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:58.772333 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a41b4e-8a35-40ad-8c96-2c0d10c63018 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:58.773767 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e7a41b4e-8a35-40ad-8c96-2c0d10c63018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:58.834527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a41b4e-8a35-40ad-8c96-2c0d10c63018 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:58.839003 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09fbff-170e-43b3-b4a9-cb1489e6c96d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:58.839687 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09fbff-170e-43b3-b4a9-cb1489e6c96d service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:58.839976 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09fbff-170e-43b3-b4a9-cb1489e6c96d service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:58.840256 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e09fbff-170e-43b3-b4a9-cb1489e6c96d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:58.842114 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7e09fbff-170e-43b3-b4a9-cb1489e6c96d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:58.905240 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09fbff-170e-43b3-b4a9-cb1489e6c96d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:58.906694 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e09fbff-170e-43b3-b4a9-cb1489e6c96d service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:58.910245 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-47037314-f1fe-4705-9776-018c73c31ffa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c48007a5a6147e5a1992f70639481c6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:58.910890 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-47037314-f1fe-4705-9776-018c73c31ffa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c48007a5a6147e5a1992f70639481c6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:58.912926 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:58.916751 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1325/2674] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:41:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:41:58.919704 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-47037314-f1fe-4705-9776-018c73c31ffa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c48007a5a6147e5a1992f70639481c6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:59.002396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-47037314-f1fe-4705-9776-018c73c31ffa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c48007a5a6147e5a1992f70639481c6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:59.003239 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-47037314-f1fe-4705-9776-018c73c31ffa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c48007a5a6147e5a1992f70639481c6'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:59.028284 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1350/2675] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:41:58 2022] DELETE /identity/v3/users/8c48007a5a6147e5a1992f70639481c6 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:59.478614 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09fbff-170e-43b3-b4a9-cb1489e6c96d service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:59.480366 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7e09fbff-170e-43b3-b4a9-cb1489e6c96d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:59.499115 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e09fbff-170e-43b3-b4a9-cb1489e6c96d service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:59.531884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:41:59.573792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac1f68-1016-4180-930b-0df5b0bf085c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:59.574702 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac1f68-1016-4180-930b-0df5b0bf085c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:59.574983 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac1f68-1016-4180-930b-0df5b0bf085c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:59.575449 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac1f68-1016-4180-930b-0df5b0bf085c admin admin] PATH_INFO: `/v3/projects/ee3b293b20ba47dfab442562c3c1c322` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:59.576696 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ac1f68-1016-4180-930b-0df5b0bf085c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee3b293b20ba47dfab442562c3c1c322)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:59.578700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ac1f68-1016-4180-930b-0df5b0bf085c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:59.690085 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-82ac1f68-1016-4180-930b-0df5b0bf085c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee3b293b20ba47dfab442562c3c1c322', 'request_id': 'req-82ac1f68-1016-4180-930b-0df5b0bf085c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:59.696614 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-82ac1f68-1016-4180-930b-0df5b0bf085c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee3b293b20ba47dfab442562c3c1c322', 'request_id': 'req-82ac1f68-1016-4180-930b-0df5b0bf085c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:59.702780 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-82ac1f68-1016-4180-930b-0df5b0bf085c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee3b293b20ba47dfab442562c3c1c322', 'request_id': 'req-82ac1f68-1016-4180-930b-0df5b0bf085c'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:59.786220 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-82ac1f68-1016-4180-930b-0df5b0bf085c admin admin] The token cache is being invalidate because project ee3b293b20ba47dfab442562c3c1c322 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:41:59.786696 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-82ac1f68-1016-4180-930b-0df5b0bf085c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:59.788108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-82ac1f68-1016-4180-930b-0df5b0bf085c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:41:59.790605 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1326/2676] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:41:59 2022] DELETE /identity/v3/projects/ee3b293b20ba47dfab442562c3c1c322 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:41:59.796872 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7b6d1-ede4-430f-964e-fc4135640094 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:59.797370 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7b6d1-ede4-430f-964e-fc4135640094 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:59.797644 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7b6d1-ede4-430f-964e-fc4135640094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:59.817252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-38b7b6d1-ede4-430f-964e-fc4135640094 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:59.818517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-38b7b6d1-ede4-430f-964e-fc4135640094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:59.885845 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1351/2677] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:59.893814 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d6272-0c34-4879-96f1-80b49dae7d34 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:59.894109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d6272-0c34-4879-96f1-80b49dae7d34 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:59.894388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d6272-0c34-4879-96f1-80b49dae7d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:59.912793 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6f1d6272-0c34-4879-96f1-80b49dae7d34 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:41:59.914545 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6f1d6272-0c34-4879-96f1-80b49dae7d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:41:59.922471 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1327/2678] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:41:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:41:59.927080 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b7b6d1-ede4-430f-964e-fc4135640094 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:59.938107 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b7b6d1-ede4-430f-964e-fc4135640094 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:59.943227 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53cb1a-d780-4307-a77a-cf3ef0b49230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:59.943947 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53cb1a-d780-4307-a77a-cf3ef0b49230 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:59.944237 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53cb1a-d780-4307-a77a-cf3ef0b49230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:59.944532 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53cb1a-d780-4307-a77a-cf3ef0b49230 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:59.945418 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53cb1a-d780-4307-a77a-cf3ef0b49230 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:41:59.945730 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53cb1a-d780-4307-a77a-cf3ef0b49230 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:59.947379 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53cb1a-d780-4307-a77a-cf3ef0b49230 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:41:59.959518 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1352/2679] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:41:59.964270 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d6272-0c34-4879-96f1-80b49dae7d34 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:41:59.974123 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d6272-0c34-4879-96f1-80b49dae7d34 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:41:59.979924 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dc80f-b429-4fca-b5d9-14e1f06c08fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:41:59.980581 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dc80f-b429-4fca-b5d9-14e1f06c08fb admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:41:59.980866 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dc80f-b429-4fca-b5d9-14e1f06c08fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:41:59.981144 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0dc80f-b429-4fca-b5d9-14e1f06c08fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:41:59.981843 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0dc80f-b429-4fca-b5d9-14e1f06c08fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:41:59.983942 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0dc80f-b429-4fca-b5d9-14e1f06c08fb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:00.067386 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1328/2680] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:41:59 2022] POST /identity/v3/projects => generated 359 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:42:00.072758 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53cb1a-d780-4307-a77a-cf3ef0b49230 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:00.083750 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53cb1a-d780-4307-a77a-cf3ef0b49230 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:00.146387 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4796b-688e-4619-931d-46d34bd617de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:00.147058 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4796b-688e-4619-931d-46d34bd617de admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:00.147343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4796b-688e-4619-931d-46d34bd617de admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:00.147651 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4796b-688e-4619-931d-46d34bd617de admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:00.149026 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4796b-688e-4619-931d-46d34bd617de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:00.150959 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4796b-688e-4619-931d-46d34bd617de admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:00.238359 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1353/2681] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:42:00 2022] POST /identity/v3/users => generated 280 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:42:00.242946 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dc80f-b429-4fca-b5d9-14e1f06c08fb admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:00.254921 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0dc80f-b429-4fca-b5d9-14e1f06c08fb admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:00.260805 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf37a8e-7d6a-4271-9b58-8b4594cea513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:00.261503 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf37a8e-7d6a-4271-9b58-8b4594cea513 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:00.261817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf37a8e-7d6a-4271-9b58-8b4594cea513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:00.262140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf37a8e-7d6a-4271-9b58-8b4594cea513 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:00.262815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf37a8e-7d6a-4271-9b58-8b4594cea513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:00.264478 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf37a8e-7d6a-4271-9b58-8b4594cea513 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:00.276359 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1329/2682] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:42:00.282183 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4796b-688e-4619-931d-46d34bd617de admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:00.294035 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4796b-688e-4619-931d-46d34bd617de admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:00.303647 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c77a9d6-a9e0-4aad-afd4-2efb38a47118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:00.304499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77a9d6-a9e0-4aad-afd4-2efb38a47118 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:00.304499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77a9d6-a9e0-4aad-afd4-2efb38a47118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:00.304499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c77a9d6-a9e0-4aad-afd4-2efb38a47118 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:00.305793 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c77a9d6-a9e0-4aad-afd4-2efb38a47118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:00.309492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c77a9d6-a9e0-4aad-afd4-2efb38a47118 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:00.325243 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1354/2683] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:42:00.328512 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf37a8e-7d6a-4271-9b58-8b4594cea513 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:00.340344 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf37a8e-7d6a-4271-9b58-8b4594cea513 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:00.345335 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e47e30-8d19-4a76-9357-53c5f110e119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:00.346223 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e47e30-8d19-4a76-9357-53c5f110e119 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:00.346508 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e47e30-8d19-4a76-9357-53c5f110e119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:00.346785 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e47e30-8d19-4a76-9357-53c5f110e119 admin admin] PATH_INFO: `/v3/projects/8404de78ee554f3ebfc7e2b8d01052d9/users/44272cbdaa324a3796bfc8b820f36818/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:00.362910 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e47e30-8d19-4a76-9357-53c5f110e119 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8404de78ee554f3ebfc7e2b8d01052d9, user_id=44272cbdaa324a3796bfc8b820f36818, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:00.365814 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e47e30-8d19-4a76-9357-53c5f110e119 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:00.473044 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1330/2684] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:42:00 2022] PUT /identity/v3/projects/8404de78ee554f3ebfc7e2b8d01052d9/users/44272cbdaa324a3796bfc8b820f36818/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:00.484637 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9968a1-9ee1-476b-804b-8d907ddbc81e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:00.484929 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9968a1-9ee1-476b-804b-8d907ddbc81e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:00.485212 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9968a1-9ee1-476b-804b-8d907ddbc81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:00.512773 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ab9968a1-9ee1-476b-804b-8d907ddbc81e None None] MFA Rules not processed for user `44272cbdaa324a3796bfc8b820f36818`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:00.514031 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ab9968a1-9ee1-476b-804b-8d907ddbc81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:00.534908 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:00.580865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1355/2685] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:00 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:00.614048 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e47e30-8d19-4a76-9357-53c5f110e119 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:00.615795 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-86e47e30-8d19-4a76-9357-53c5f110e119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:00.683024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e47e30-8d19-4a76-9357-53c5f110e119 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:00.686713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c2863-4579-4343-b6c4-1b27d9f644c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:00.687365 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c2863-4579-4343-b6c4-1b27d9f644c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:00.687703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c2863-4579-4343-b6c4-1b27d9f644c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:00.687981 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c2863-4579-4343-b6c4-1b27d9f644c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:00.697606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c2863-4579-4343-b6c4-1b27d9f644c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:00.699069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c2863-4579-4343-b6c4-1b27d9f644c9 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:00.703839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:00.708789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1331/2686] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:42:00 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:42:00.725985 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9968a1-9ee1-476b-804b-8d907ddbc81e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:00.795590 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9968a1-9ee1-476b-804b-8d907ddbc81e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:00.800117 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ccc196-5e34-40b8-838e-61bf4ee7d298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:00.800985 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccc196-5e34-40b8-838e-61bf4ee7d298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:00.801265 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccc196-5e34-40b8-838e-61bf4ee7d298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:00.801541 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ccc196-5e34-40b8-838e-61bf4ee7d298 admin admin] PATH_INFO: `/v3/users/44272cbdaa324a3796bfc8b820f36818` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:00.802857 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ccc196-5e34-40b8-838e-61bf4ee7d298 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44272cbdaa324a3796bfc8b820f36818)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:00.805375 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ccc196-5e34-40b8-838e-61bf4ee7d298 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:01.078628 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-69ccc196-5e34-40b8-838e-61bf4ee7d298 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44272cbdaa324a3796bfc8b820f36818'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:01.079509 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-69ccc196-5e34-40b8-838e-61bf4ee7d298 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44272cbdaa324a3796bfc8b820f36818'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:01.087377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-69ccc196-5e34-40b8-838e-61bf4ee7d298 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44272cbdaa324a3796bfc8b820f36818'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:01.192620 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-69ccc196-5e34-40b8-838e-61bf4ee7d298 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44272cbdaa324a3796bfc8b820f36818'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:01.193343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-69ccc196-5e34-40b8-838e-61bf4ee7d298 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44272cbdaa324a3796bfc8b820f36818'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:01.212742 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1356/2687] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:42:00 2022] DELETE /identity/v3/users/44272cbdaa324a3796bfc8b820f36818 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:01.219347 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d4470-672e-4259-be83-72b795be10e3 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:01.219663 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d4470-672e-4259-be83-72b795be10e3 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:01.219942 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d4470-672e-4259-be83-72b795be10e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:01.238723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-fd2d4470-672e-4259-be83-72b795be10e3 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:01.240005 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fd2d4470-672e-4259-be83-72b795be10e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:01.302623 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1332/2688] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:01.313204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ccc196-5e34-40b8-838e-61bf4ee7d298 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:01.314680 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-69ccc196-5e34-40b8-838e-61bf4ee7d298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:01.362588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:01.375279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ccc196-5e34-40b8-838e-61bf4ee7d298 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:01.379030 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d3717-933b-4739-971a-56924d0f9d1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:01.379707 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d3717-933b-4739-971a-56924d0f9d1c service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:01.379993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d3717-933b-4739-971a-56924d0f9d1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:01.380272 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d3717-933b-4739-971a-56924d0f9d1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:01.389316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d3717-933b-4739-971a-56924d0f9d1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:01.390863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d3717-933b-4739-971a-56924d0f9d1c service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:01.400805 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1357/2689] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:42:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:42:01.851922 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d4470-672e-4259-be83-72b795be10e3 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:01.855079 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fd2d4470-672e-4259-be83-72b795be10e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:01.866411 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d4470-672e-4259-be83-72b795be10e3 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:01.872248 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f148d-c2cc-46c8-9a23-db4d6c128e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:01.873127 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f148d-c2cc-46c8-9a23-db4d6c128e89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:01.873410 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f148d-c2cc-46c8-9a23-db4d6c128e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:01.873709 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f148d-c2cc-46c8-9a23-db4d6c128e89 admin admin] PATH_INFO: `/v3/projects/8404de78ee554f3ebfc7e2b8d01052d9` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:01.874976 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f148d-c2cc-46c8-9a23-db4d6c128e89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8404de78ee554f3ebfc7e2b8d01052d9)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:01.876955 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f148d-c2cc-46c8-9a23-db4d6c128e89 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:02.010567 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ba6f148d-c2cc-46c8-9a23-db4d6c128e89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8404de78ee554f3ebfc7e2b8d01052d9', 'request_id': 'req-ba6f148d-c2cc-46c8-9a23-db4d6c128e89'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:02.016826 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ba6f148d-c2cc-46c8-9a23-db4d6c128e89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8404de78ee554f3ebfc7e2b8d01052d9', 'request_id': 'req-ba6f148d-c2cc-46c8-9a23-db4d6c128e89'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:02.022451 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ba6f148d-c2cc-46c8-9a23-db4d6c128e89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8404de78ee554f3ebfc7e2b8d01052d9', 'request_id': 'req-ba6f148d-c2cc-46c8-9a23-db4d6c128e89'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:02.106212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ba6f148d-c2cc-46c8-9a23-db4d6c128e89 admin admin] The token cache is being invalidate because project 8404de78ee554f3ebfc7e2b8d01052d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:42:02.106505 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ba6f148d-c2cc-46c8-9a23-db4d6c128e89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:02.107142 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ba6f148d-c2cc-46c8-9a23-db4d6c128e89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:02.108952 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1333/2690] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:42:01 2022] DELETE /identity/v3/projects/8404de78ee554f3ebfc7e2b8d01052d9 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:02.113870 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18317c-132e-4080-bdd8-10b97859a91d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:02.114351 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18317c-132e-4080-bdd8-10b97859a91d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:02.114466 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18317c-132e-4080-bdd8-10b97859a91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:02.135320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5f18317c-132e-4080-bdd8-10b97859a91d None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:02.136573 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5f18317c-132e-4080-bdd8-10b97859a91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:02.201574 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1358/2691] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:02.209406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a91bd-ebc9-45a2-a7c9-18e10c231656 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:02.209702 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a91bd-ebc9-45a2-a7c9-18e10c231656 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:02.209990 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a91bd-ebc9-45a2-a7c9-18e10c231656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:02.235190 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-676a91bd-ebc9-45a2-a7c9-18e10c231656 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:02.236471 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-676a91bd-ebc9-45a2-a7c9-18e10c231656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:02.245651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1334/2692] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:02.249944 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18317c-132e-4080-bdd8-10b97859a91d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:02.259993 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18317c-132e-4080-bdd8-10b97859a91d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:02.265294 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4cbd6-ff5c-4598-8256-26d452cd1414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:02.266018 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4cbd6-ff5c-4598-8256-26d452cd1414 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:02.271941 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4cbd6-ff5c-4598-8256-26d452cd1414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:02.271941 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4cbd6-ff5c-4598-8256-26d452cd1414 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:02.271941 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4cbd6-ff5c-4598-8256-26d452cd1414 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:42:02.271941 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4cbd6-ff5c-4598-8256-26d452cd1414 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:02.271941 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4cbd6-ff5c-4598-8256-26d452cd1414 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:02.282409 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1359/2693] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:42:02.286591 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a91bd-ebc9-45a2-a7c9-18e10c231656 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:02.296602 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a91bd-ebc9-45a2-a7c9-18e10c231656 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:02.301289 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6fd28b-9e43-4055-b2b2-98b5fc596f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:02.301945 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fd28b-9e43-4055-b2b2-98b5fc596f71 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:02.302227 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fd28b-9e43-4055-b2b2-98b5fc596f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:02.302506 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6fd28b-9e43-4055-b2b2-98b5fc596f71 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:02.303235 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6fd28b-9e43-4055-b2b2-98b5fc596f71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:02.305239 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6fd28b-9e43-4055-b2b2-98b5fc596f71 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:02.394808 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1335/2694] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:42:02 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:42:02.398454 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4cbd6-ff5c-4598-8256-26d452cd1414 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:02.409508 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4cbd6-ff5c-4598-8256-26d452cd1414 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:02.467084 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062770c9-10ae-4de8-825b-ea8f42f272ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:02.467790 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062770c9-10ae-4de8-825b-ea8f42f272ca admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:02.468080 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062770c9-10ae-4de8-825b-ea8f42f272ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:02.468363 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062770c9-10ae-4de8-825b-ea8f42f272ca admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:02.469067 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062770c9-10ae-4de8-825b-ea8f42f272ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:02.471001 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062770c9-10ae-4de8-825b-ea8f42f272ca admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:02.554580 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1360/2695] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:42:02 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:42:02.556453 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6fd28b-9e43-4055-b2b2-98b5fc596f71 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:02.567456 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6fd28b-9e43-4055-b2b2-98b5fc596f71 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:02.572443 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5afba-2e75-4a7c-a189-4423475e5a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:02.573101 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5afba-2e75-4a7c-a189-4423475e5a45 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:02.573394 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5afba-2e75-4a7c-a189-4423475e5a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:02.573671 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5afba-2e75-4a7c-a189-4423475e5a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:02.574394 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d5afba-2e75-4a7c-a189-4423475e5a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:02.576136 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d5afba-2e75-4a7c-a189-4423475e5a45 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:02.586900 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1336/2696] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:42:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:42:02.591626 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062770c9-10ae-4de8-825b-ea8f42f272ca admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:02.601851 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062770c9-10ae-4de8-825b-ea8f42f272ca admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:02.607258 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9cf178-fb83-44e2-b075-99c8fab06b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:02.607959 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cf178-fb83-44e2-b075-99c8fab06b78 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:02.608274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cf178-fb83-44e2-b075-99c8fab06b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:02.608556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9cf178-fb83-44e2-b075-99c8fab06b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:02.609283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9cf178-fb83-44e2-b075-99c8fab06b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:02.610934 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9cf178-fb83-44e2-b075-99c8fab06b78 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:02.621264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1361/2697] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:42:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:42:02.625402 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5afba-2e75-4a7c-a189-4423475e5a45 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:02.635921 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5afba-2e75-4a7c-a189-4423475e5a45 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:02.640679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be9d4e-4c09-47f9-97f8-d8807265ee62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:02.641786 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be9d4e-4c09-47f9-97f8-d8807265ee62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:02.641893 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be9d4e-4c09-47f9-97f8-d8807265ee62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:02.642125 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84be9d4e-4c09-47f9-97f8-d8807265ee62 admin admin] PATH_INFO: `/v3/projects/450b26a225464051b4c488c847ae87f5/users/3579fb153d3340c282ddffab5d3c6203/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:02.659458 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be9d4e-4c09-47f9-97f8-d8807265ee62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=450b26a225464051b4c488c847ae87f5, user_id=3579fb153d3340c282ddffab5d3c6203, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:02.662354 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84be9d4e-4c09-47f9-97f8-d8807265ee62 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:02.776445 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1337/2698] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:42:02 2022] PUT /identity/v3/projects/450b26a225464051b4c488c847ae87f5/users/3579fb153d3340c282ddffab5d3c6203/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:02.787020 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b83c4-ef72-4b9c-83fd-167c378f1157 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:02.787315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b83c4-ef72-4b9c-83fd-167c378f1157 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:02.787630 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b83c4-ef72-4b9c-83fd-167c378f1157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:02.815805 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-a85b83c4-ef72-4b9c-83fd-167c378f1157 None None] MFA Rules not processed for user `3579fb153d3340c282ddffab5d3c6203`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:02.817314 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a85b83c4-ef72-4b9c-83fd-167c378f1157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:02.863148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:02.889440 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1362/2699] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:02.894079 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be9d4e-4c09-47f9-97f8-d8807265ee62 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:02.916633 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:02.958166 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84be9d4e-4c09-47f9-97f8-d8807265ee62 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:02.963849 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf91bc-65d1-455f-803c-4d1fddb80bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:02.964552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf91bc-65d1-455f-803c-4d1fddb80bb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:02.964880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf91bc-65d1-455f-803c-4d1fddb80bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:02.965221 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf91bc-65d1-455f-803c-4d1fddb80bb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:02.965948 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf91bc-65d1-455f-803c-4d1fddb80bb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:02.968066 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf91bc-65d1-455f-803c-4d1fddb80bb3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:03.108587 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1338/2700] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:42:02 2022] POST /identity/v3/projects => generated 377 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:42:03.112644 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b83c4-ef72-4b9c-83fd-167c378f1157 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:03.124647 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b83c4-ef72-4b9c-83fd-167c378f1157 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:03.183960 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6824b2-7cc8-4b38-bf88-ecfdd029e7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:03.184570 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6824b2-7cc8-4b38-bf88-ecfdd029e7cd admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:03.184862 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6824b2-7cc8-4b38-bf88-ecfdd029e7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:03.185144 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6824b2-7cc8-4b38-bf88-ecfdd029e7cd admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:03.185856 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6824b2-7cc8-4b38-bf88-ecfdd029e7cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:03.187973 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6824b2-7cc8-4b38-bf88-ecfdd029e7cd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:03.299661 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1363/2701] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:42:03 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:42:03.318485 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf91bc-65d1-455f-803c-4d1fddb80bb3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:03.329573 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf91bc-65d1-455f-803c-4d1fddb80bb3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:03.334324 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a127036-65c0-4fc0-b609-ad1c20304551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:03.335081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a127036-65c0-4fc0-b609-ad1c20304551 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:03.335365 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a127036-65c0-4fc0-b609-ad1c20304551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:03.335745 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a127036-65c0-4fc0-b609-ad1c20304551 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:03.336480 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a127036-65c0-4fc0-b609-ad1c20304551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:03.338076 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a127036-65c0-4fc0-b609-ad1c20304551 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:03.349260 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1339/2702] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:42:03.353329 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6824b2-7cc8-4b38-bf88-ecfdd029e7cd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:03.364568 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6824b2-7cc8-4b38-bf88-ecfdd029e7cd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:03.369443 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7359ce-09b0-4183-9de8-2865445ee632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:03.370779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7359ce-09b0-4183-9de8-2865445ee632 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:03.371058 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7359ce-09b0-4183-9de8-2865445ee632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:03.371333 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7359ce-09b0-4183-9de8-2865445ee632 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:03.372094 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7359ce-09b0-4183-9de8-2865445ee632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:03.373701 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7359ce-09b0-4183-9de8-2865445ee632 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:03.389740 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1364/2703] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:42:03.392680 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a127036-65c0-4fc0-b609-ad1c20304551 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:03.403005 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a127036-65c0-4fc0-b609-ad1c20304551 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:03.407701 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b0969-d882-49f6-ae58-0c03a1e756e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:03.408588 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b0969-d882-49f6-ae58-0c03a1e756e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:03.408873 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b0969-d882-49f6-ae58-0c03a1e756e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:03.409150 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b0969-d882-49f6-ae58-0c03a1e756e4 admin admin] PATH_INFO: `/v3/projects/2a9c1b34b4074e0081684b2dc1e98182/users/3818552a1d1347e8a767c93e22e49d8a/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:03.424344 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3b0969-d882-49f6-ae58-0c03a1e756e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a9c1b34b4074e0081684b2dc1e98182, user_id=3818552a1d1347e8a767c93e22e49d8a, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:03.427076 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3b0969-d882-49f6-ae58-0c03a1e756e4 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:03.667695 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1340/2704] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:42:03 2022] PUT /identity/v3/projects/2a9c1b34b4074e0081684b2dc1e98182/users/3818552a1d1347e8a767c93e22e49d8a/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:03.679091 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5bd1c-7a56-466c-8e18-a17b595252e4 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:03.679386 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5bd1c-7a56-466c-8e18-a17b595252e4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:03.679731 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5bd1c-7a56-466c-8e18-a17b595252e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:03.707918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3fc5bd1c-7a56-466c-8e18-a17b595252e4 None None] MFA Rules not processed for user `3818552a1d1347e8a767c93e22e49d8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:03.709141 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3fc5bd1c-7a56-466c-8e18-a17b595252e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:03.779802 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1365/2705] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:03.789708 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b0969-d882-49f6-ae58-0c03a1e756e4 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:03.856666 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b0969-d882-49f6-ae58-0c03a1e756e4 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:03.861791 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50620f03-39d2-475f-be38-7ec6b3b0e1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:03.862479 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50620f03-39d2-475f-be38-7ec6b3b0e1be admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:03.862759 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50620f03-39d2-475f-be38-7ec6b3b0e1be admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:03.863030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50620f03-39d2-475f-be38-7ec6b3b0e1be admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:03.863769 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50620f03-39d2-475f-be38-7ec6b3b0e1be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:03.865764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50620f03-39d2-475f-be38-7ec6b3b0e1be admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:04.086280 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1341/2706] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:42:03 2022] POST /identity/v3/projects => generated 373 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:42:04.091012 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5bd1c-7a56-466c-8e18-a17b595252e4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:04.093349 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50620f03-39d2-475f-be38-7ec6b3b0e1be admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:04.095233 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-50620f03-39d2-475f-be38-7ec6b3b0e1be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:04.103266 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5bd1c-7a56-466c-8e18-a17b595252e4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:04.149520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:04.154989 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50620f03-39d2-475f-be38-7ec6b3b0e1be admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:04.158976 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda21744-2771-41b9-bfed-ebbf2b059580 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:04.159666 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda21744-2771-41b9-bfed-ebbf2b059580 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:04.159952 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda21744-2771-41b9-bfed-ebbf2b059580 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:04.160231 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda21744-2771-41b9-bfed-ebbf2b059580 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:04.161995 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fda21744-2771-41b9-bfed-ebbf2b059580 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:04.162495 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b706c63-67bf-4ec1-91d7-74ab2f0b7004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:04.163223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b706c63-67bf-4ec1-91d7-74ab2f0b7004 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:04.163538 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b706c63-67bf-4ec1-91d7-74ab2f0b7004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:04.163827 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b706c63-67bf-4ec1-91d7-74ab2f0b7004 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:04.164523 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b706c63-67bf-4ec1-91d7-74ab2f0b7004 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:04.166518 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b706c63-67bf-4ec1-91d7-74ab2f0b7004 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:04.172830 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:04.222833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda21744-2771-41b9-bfed-ebbf2b059580 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:04.224258 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda21744-2771-41b9-bfed-ebbf2b059580 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:04.234208 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1342/2707] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:42:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:42:04.249931 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1366/2708] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:42:04 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:42:04.254100 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda21744-2771-41b9-bfed-ebbf2b059580 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:04.266278 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda21744-2771-41b9-bfed-ebbf2b059580 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:04.270838 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37924347-6a3e-4810-9bc2-fc84b67c6914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:04.271558 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37924347-6a3e-4810-9bc2-fc84b67c6914 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:04.271842 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37924347-6a3e-4810-9bc2-fc84b67c6914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:04.272140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37924347-6a3e-4810-9bc2-fc84b67c6914 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:04.272861 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37924347-6a3e-4810-9bc2-fc84b67c6914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:04.274449 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37924347-6a3e-4810-9bc2-fc84b67c6914 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:04.285582 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1343/2709] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:42:04.290157 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b706c63-67bf-4ec1-91d7-74ab2f0b7004 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:04.302135 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b706c63-67bf-4ec1-91d7-74ab2f0b7004 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:04.308926 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a76cc-2f76-4ec9-8217-9a6edb5c75fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:04.310597 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a76cc-2f76-4ec9-8217-9a6edb5c75fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:04.310960 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a76cc-2f76-4ec9-8217-9a6edb5c75fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:04.311336 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a76cc-2f76-4ec9-8217-9a6edb5c75fa admin admin] PATH_INFO: `/v3/projects/11e37a9637df45ecbba5343b6022407d/users/123f35fed9534b7f8d4441bc65bcb2ef/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:04.333252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a76cc-2f76-4ec9-8217-9a6edb5c75fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=11e37a9637df45ecbba5343b6022407d, user_id=123f35fed9534b7f8d4441bc65bcb2ef, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:04.336058 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a76cc-2f76-4ec9-8217-9a6edb5c75fa admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:04.415640 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1367/2710] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:42:04 2022] PUT /identity/v3/projects/11e37a9637df45ecbba5343b6022407d/users/123f35fed9534b7f8d4441bc65bcb2ef/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:04.420387 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37924347-6a3e-4810-9bc2-fc84b67c6914 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:04.430361 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37924347-6a3e-4810-9bc2-fc84b67c6914 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:04.490170 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39721d95-5049-4f10-b4a5-71fde23af39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:04.490766 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39721d95-5049-4f10-b4a5-71fde23af39d admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:04.491051 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39721d95-5049-4f10-b4a5-71fde23af39d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:04.491332 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39721d95-5049-4f10-b4a5-71fde23af39d admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:04.492081 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39721d95-5049-4f10-b4a5-71fde23af39d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:04.493690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39721d95-5049-4f10-b4a5-71fde23af39d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:04.503576 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1344/2711] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:42:04.507980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a76cc-2f76-4ec9-8217-9a6edb5c75fa admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:04.519195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a76cc-2f76-4ec9-8217-9a6edb5c75fa admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:04.523994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d1e02-f2b4-46ed-856d-3de728c1c8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:04.524869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d1e02-f2b4-46ed-856d-3de728c1c8c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:04.525149 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d1e02-f2b4-46ed-856d-3de728c1c8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:04.525425 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d1e02-f2b4-46ed-856d-3de728c1c8c7 admin admin] PATH_INFO: `/v3/projects/11e37a9637df45ecbba5343b6022407d/users/123f35fed9534b7f8d4441bc65bcb2ef/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:04.528032 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d1e02-f2b4-46ed-856d-3de728c1c8c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11e37a9637df45ecbba5343b6022407d, user_id=123f35fed9534b7f8d4441bc65bcb2ef, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:04.530688 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d1e02-f2b4-46ed-856d-3de728c1c8c7 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:04.541058 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1368/2712] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:42:04 2022] PUT /identity/v3/projects/11e37a9637df45ecbba5343b6022407d/users/123f35fed9534b7f8d4441bc65bcb2ef/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:04.551645 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5369de-6b41-4da6-945c-ee6c36d42962 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:04.551937 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5369de-6b41-4da6-945c-ee6c36d42962 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:04.552220 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5369de-6b41-4da6-945c-ee6c36d42962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:04.580690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-ce5369de-6b41-4da6-945c-ee6c36d42962 None None] MFA Rules not processed for user `123f35fed9534b7f8d4441bc65bcb2ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:04.581912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ce5369de-6b41-4da6-945c-ee6c36d42962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:04.651229 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1345/2713] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:04 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:04.663533 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d1e02-f2b4-46ed-856d-3de728c1c8c7 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:04.665331 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-464d1e02-f2b4-46ed-856d-3de728c1c8c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:04.728617 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d1e02-f2b4-46ed-856d-3de728c1c8c7 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:04.732744 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c89c4a-63e3-4b87-89cc-181c31fb0971 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:04.733394 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c89c4a-63e3-4b87-89cc-181c31fb0971 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:04.733686 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c89c4a-63e3-4b87-89cc-181c31fb0971 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:04.733965 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c89c4a-63e3-4b87-89cc-181c31fb0971 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:04.792466 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c89c4a-63e3-4b87-89cc-181c31fb0971 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:04.797478 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c89c4a-63e3-4b87-89cc-181c31fb0971 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:04.806626 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1369/2714] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:42:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:42:06.864461 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5369de-6b41-4da6-945c-ee6c36d42962 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:06.875606 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5369de-6b41-4da6-945c-ee6c36d42962 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:06.879265 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abcc83-e8d8-4439-b127-b4eb865f6c16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:06.879983 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abcc83-e8d8-4439-b127-b4eb865f6c16 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:06.880274 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abcc83-e8d8-4439-b127-b4eb865f6c16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:06.880567 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abcc83-e8d8-4439-b127-b4eb865f6c16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:06.897921 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:06.938604 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abcc83-e8d8-4439-b127-b4eb865f6c16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:06.940102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abcc83-e8d8-4439-b127-b4eb865f6c16 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:06.950136 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1346/2715] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:42:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:42:08.477400 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c89c4a-63e3-4b87-89cc-181c31fb0971 service cinder] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:08.479327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-41c89c4a-63e3-4b87-89cc-181c31fb0971 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:08.562699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:08.567915 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abcc83-e8d8-4439-b127-b4eb865f6c16 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:08.569838 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c89c4a-63e3-4b87-89cc-181c31fb0971 service cinder] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:08.574998 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce5477-d1fa-4f0b-b692-106b5279736b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:08.576312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce5477-d1fa-4f0b-b692-106b5279736b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:08.576424 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce5477-d1fa-4f0b-b692-106b5279736b admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:08.576788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce5477-d1fa-4f0b-b692-106b5279736b admin admin] PATH_INFO: `/v3/users/c628648fa7fd4d13a96235051c87cfbb` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:08.577435 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abcc83-e8d8-4439-b127-b4eb865f6c16 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:08.578184 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce5477-d1fa-4f0b-b692-106b5279736b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c628648fa7fd4d13a96235051c87cfbb)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:08.580359 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce5477-d1fa-4f0b-b692-106b5279736b admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:08.581226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9847e2b0-ed7c-4af4-b22f-a9b2d15fae8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:08.581875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847e2b0-ed7c-4af4-b22f-a9b2d15fae8b service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:08.582163 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847e2b0-ed7c-4af4-b22f-a9b2d15fae8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:08.582446 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847e2b0-ed7c-4af4-b22f-a9b2d15fae8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:08.594327 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9847e2b0-ed7c-4af4-b22f-a9b2d15fae8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:08.595718 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9847e2b0-ed7c-4af4-b22f-a9b2d15fae8b service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:08.606059 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1347/2716] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:42:08 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:42:08.808806 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-46ce5477-d1fa-4f0b-b692-106b5279736b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c628648fa7fd4d13a96235051c87cfbb'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:08.809607 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-46ce5477-d1fa-4f0b-b692-106b5279736b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c628648fa7fd4d13a96235051c87cfbb'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:08.817124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-46ce5477-d1fa-4f0b-b692-106b5279736b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c628648fa7fd4d13a96235051c87cfbb'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:08.900770 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-46ce5477-d1fa-4f0b-b692-106b5279736b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c628648fa7fd4d13a96235051c87cfbb'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:08.901409 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-46ce5477-d1fa-4f0b-b692-106b5279736b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c628648fa7fd4d13a96235051c87cfbb'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:08.921279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1370/2717] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:42:08 2022] DELETE /identity/v3/users/c628648fa7fd4d13a96235051c87cfbb => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:08.925851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c6141-ec9a-47a1-8b0b-dccf2256c007 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:08.926138 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c6141-ec9a-47a1-8b0b-dccf2256c007 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:08.926419 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c6141-ec9a-47a1-8b0b-dccf2256c007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:08.944239 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-e72c6141-ec9a-47a1-8b0b-dccf2256c007 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:08.945442 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e72c6141-ec9a-47a1-8b0b-dccf2256c007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:09.006428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1348/2718] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:09.017208 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce5477-d1fa-4f0b-b692-106b5279736b admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:09.018214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-46ce5477-d1fa-4f0b-b692-106b5279736b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:09.077279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce5477-d1fa-4f0b-b692-106b5279736b admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:09.081056 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8001c1f-778f-40cd-bcce-9937379cec07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:09.081729 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8001c1f-778f-40cd-bcce-9937379cec07 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:09.081994 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8001c1f-778f-40cd-bcce-9937379cec07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:09.082272 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8001c1f-778f-40cd-bcce-9937379cec07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:09.092469 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8001c1f-778f-40cd-bcce-9937379cec07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:09.093940 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8001c1f-778f-40cd-bcce-9937379cec07 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:09.104165 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1371/2719] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:42:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:42:09.588665 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c6141-ec9a-47a1-8b0b-dccf2256c007 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:09.590190 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e72c6141-ec9a-47a1-8b0b-dccf2256c007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:09.603785 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c6141-ec9a-47a1-8b0b-dccf2256c007 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:09.608947 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c1503-3afb-4feb-9ee8-68d5fda9c8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:09.609923 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c1503-3afb-4feb-9ee8-68d5fda9c8dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:09.610730 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c1503-3afb-4feb-9ee8-68d5fda9c8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:09.611020 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c1503-3afb-4feb-9ee8-68d5fda9c8dd admin admin] PATH_INFO: `/v3/projects/b1d795ffa7f44314b3a74ec658117645` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:09.612608 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c1503-3afb-4feb-9ee8-68d5fda9c8dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1d795ffa7f44314b3a74ec658117645)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:09.614610 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c1503-3afb-4feb-9ee8-68d5fda9c8dd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:09.714300 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-990c1503-3afb-4feb-9ee8-68d5fda9c8dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1d795ffa7f44314b3a74ec658117645', 'request_id': 'req-990c1503-3afb-4feb-9ee8-68d5fda9c8dd'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:09.720942 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-990c1503-3afb-4feb-9ee8-68d5fda9c8dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1d795ffa7f44314b3a74ec658117645', 'request_id': 'req-990c1503-3afb-4feb-9ee8-68d5fda9c8dd'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:09.726685 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-990c1503-3afb-4feb-9ee8-68d5fda9c8dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1d795ffa7f44314b3a74ec658117645', 'request_id': 'req-990c1503-3afb-4feb-9ee8-68d5fda9c8dd'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:09.832640 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-990c1503-3afb-4feb-9ee8-68d5fda9c8dd admin admin] The token cache is being invalidate because project b1d795ffa7f44314b3a74ec658117645 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:42:09.832947 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-990c1503-3afb-4feb-9ee8-68d5fda9c8dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:09.833551 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-990c1503-3afb-4feb-9ee8-68d5fda9c8dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:09.835504 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1349/2720] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:42:09 2022] DELETE /identity/v3/projects/b1d795ffa7f44314b3a74ec658117645 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:09.839288 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8001c1f-778f-40cd-bcce-9937379cec07 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:09.849214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8001c1f-778f-40cd-bcce-9937379cec07 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:09.910747 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d3599-eae2-4d02-8f83-23c7e48f05a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:09.911788 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d3599-eae2-4d02-8f83-23c7e48f05a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:09.912075 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d3599-eae2-4d02-8f83-23c7e48f05a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:09.912355 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d3599-eae2-4d02-8f83-23c7e48f05a4 admin admin] PATH_INFO: `/v3/users/ac82df04bc7949b5a519a38e3d37285d` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:09.913581 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d3599-eae2-4d02-8f83-23c7e48f05a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac82df04bc7949b5a519a38e3d37285d)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:09.918224 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d3599-eae2-4d02-8f83-23c7e48f05a4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:09.946653 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:10.201847 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5b3d3599-eae2-4d02-8f83-23c7e48f05a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac82df04bc7949b5a519a38e3d37285d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:10.202621 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5b3d3599-eae2-4d02-8f83-23c7e48f05a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac82df04bc7949b5a519a38e3d37285d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:10.210316 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5b3d3599-eae2-4d02-8f83-23c7e48f05a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac82df04bc7949b5a519a38e3d37285d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:10.304874 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5b3d3599-eae2-4d02-8f83-23c7e48f05a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac82df04bc7949b5a519a38e3d37285d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:10.305523 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5b3d3599-eae2-4d02-8f83-23c7e48f05a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac82df04bc7949b5a519a38e3d37285d'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:10.324656 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1372/2721] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:42:09 2022] DELETE /identity/v3/users/ac82df04bc7949b5a519a38e3d37285d => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:42:10.784367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c1503-3afb-4feb-9ee8-68d5fda9c8dd admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:10.792386 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-990c1503-3afb-4feb-9ee8-68d5fda9c8dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:10.799841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c1503-3afb-4feb-9ee8-68d5fda9c8dd admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:10.837424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:10.872371 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8bc5c4-6e20-481c-8997-c65cd7cf9879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:10.873320 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bc5c4-6e20-481c-8997-c65cd7cf9879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:10.873609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bc5c4-6e20-481c-8997-c65cd7cf9879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:10.873884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8bc5c4-6e20-481c-8997-c65cd7cf9879 admin admin] PATH_INFO: `/v3/projects/66af0479fdf84b8bae47494cdef54f53` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:10.875134 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8bc5c4-6e20-481c-8997-c65cd7cf9879 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66af0479fdf84b8bae47494cdef54f53)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:10.877312 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8bc5c4-6e20-481c-8997-c65cd7cf9879 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:11.025871 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fc8bc5c4-6e20-481c-8997-c65cd7cf9879 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66af0479fdf84b8bae47494cdef54f53', 'request_id': 'req-fc8bc5c4-6e20-481c-8997-c65cd7cf9879'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:11.032431 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fc8bc5c4-6e20-481c-8997-c65cd7cf9879 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66af0479fdf84b8bae47494cdef54f53', 'request_id': 'req-fc8bc5c4-6e20-481c-8997-c65cd7cf9879'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:11.037939 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fc8bc5c4-6e20-481c-8997-c65cd7cf9879 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66af0479fdf84b8bae47494cdef54f53', 'request_id': 'req-fc8bc5c4-6e20-481c-8997-c65cd7cf9879'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:11.122946 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fc8bc5c4-6e20-481c-8997-c65cd7cf9879 admin admin] The token cache is being invalidate because project 66af0479fdf84b8bae47494cdef54f53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:42:11.123236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fc8bc5c4-6e20-481c-8997-c65cd7cf9879 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:11.123894 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-fc8bc5c4-6e20-481c-8997-c65cd7cf9879 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:11.125459 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1350/2722] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:42:10 2022] DELETE /identity/v3/projects/66af0479fdf84b8bae47494cdef54f53 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:11.129900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d3599-eae2-4d02-8f83-23c7e48f05a4 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:11.131325 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5b3d3599-eae2-4d02-8f83-23c7e48f05a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:11.145285 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d3599-eae2-4d02-8f83-23c7e48f05a4 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:11.205165 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a80508-04c9-4750-93c2-fdd9c6e1e808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:11.206041 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a80508-04c9-4750-93c2-fdd9c6e1e808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:11.206322 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a80508-04c9-4750-93c2-fdd9c6e1e808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:11.206596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a80508-04c9-4750-93c2-fdd9c6e1e808 admin admin] PATH_INFO: `/v3/users/1b649c84b1a744ba89af7f2b03a8917e` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:11.207826 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a80508-04c9-4750-93c2-fdd9c6e1e808 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b649c84b1a744ba89af7f2b03a8917e)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:11.209739 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a80508-04c9-4750-93c2-fdd9c6e1e808 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:11.355229 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-40a80508-04c9-4750-93c2-fdd9c6e1e808 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b649c84b1a744ba89af7f2b03a8917e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:11.355914 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-40a80508-04c9-4750-93c2-fdd9c6e1e808 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b649c84b1a744ba89af7f2b03a8917e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:11.364195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-40a80508-04c9-4750-93c2-fdd9c6e1e808 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b649c84b1a744ba89af7f2b03a8917e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:11.409753 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-40a80508-04c9-4750-93c2-fdd9c6e1e808 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b649c84b1a744ba89af7f2b03a8917e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:11.410390 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-40a80508-04c9-4750-93c2-fdd9c6e1e808 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b649c84b1a744ba89af7f2b03a8917e'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:11.429659 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1373/2723] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:42:11 2022] DELETE /identity/v3/users/1b649c84b1a744ba89af7f2b03a8917e => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:11.928063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8bc5c4-6e20-481c-8997-c65cd7cf9879 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:11.930342 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-fc8bc5c4-6e20-481c-8997-c65cd7cf9879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:11.945455 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8bc5c4-6e20-481c-8997-c65cd7cf9879 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:12.006845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c559e-ee7b-4bcc-8b4e-7db61aa1308a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:12.007756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c559e-ee7b-4bcc-8b4e-7db61aa1308a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:12.008038 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c559e-ee7b-4bcc-8b4e-7db61aa1308a admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:12.008339 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c559e-ee7b-4bcc-8b4e-7db61aa1308a admin admin] PATH_INFO: `/v3/projects/5d1099f456ea4f369f34257aed5009e1` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:12.009561 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881c559e-ee7b-4bcc-8b4e-7db61aa1308a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d1099f456ea4f369f34257aed5009e1)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:12.011517 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881c559e-ee7b-4bcc-8b4e-7db61aa1308a admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:12.177009 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-881c559e-ee7b-4bcc-8b4e-7db61aa1308a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d1099f456ea4f369f34257aed5009e1', 'request_id': 'req-881c559e-ee7b-4bcc-8b4e-7db61aa1308a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:12.177197 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-881c559e-ee7b-4bcc-8b4e-7db61aa1308a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d1099f456ea4f369f34257aed5009e1', 'request_id': 'req-881c559e-ee7b-4bcc-8b4e-7db61aa1308a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:12.182009 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:12.183881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-881c559e-ee7b-4bcc-8b4e-7db61aa1308a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d1099f456ea4f369f34257aed5009e1', 'request_id': 'req-881c559e-ee7b-4bcc-8b4e-7db61aa1308a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:12.278973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-881c559e-ee7b-4bcc-8b4e-7db61aa1308a admin admin] The token cache is being invalidate because project 5d1099f456ea4f369f34257aed5009e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:42:12.279410 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-881c559e-ee7b-4bcc-8b4e-7db61aa1308a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:12.280131 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-881c559e-ee7b-4bcc-8b4e-7db61aa1308a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:12.281808 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1351/2724] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:42:11 2022] DELETE /identity/v3/projects/5d1099f456ea4f369f34257aed5009e1 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:12.288232 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73922776-a45b-4d27-9343-b89ad10be45a None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:12.289118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73922776-a45b-4d27-9343-b89ad10be45a None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:12.289118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73922776-a45b-4d27-9343-b89ad10be45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:12.307277 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-73922776-a45b-4d27-9343-b89ad10be45a None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:12.309359 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-73922776-a45b-4d27-9343-b89ad10be45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:12.318821 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:12.369264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1374/2725] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:12.380094 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634be635-e26b-402f-93f9-4ba985fe2d08 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:12.380324 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634be635-e26b-402f-93f9-4ba985fe2d08 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:12.380600 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634be635-e26b-402f-93f9-4ba985fe2d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:12.400764 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-634be635-e26b-402f-93f9-4ba985fe2d08 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:12.402011 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-634be635-e26b-402f-93f9-4ba985fe2d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:12.409921 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1352/2726] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:12.414156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73922776-a45b-4d27-9343-b89ad10be45a None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:12.425163 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73922776-a45b-4d27-9343-b89ad10be45a None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:12.432412 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d3779-9835-415f-8221-86eb165ff2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:12.433076 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d3779-9835-415f-8221-86eb165ff2e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:12.433360 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d3779-9835-415f-8221-86eb165ff2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:12.433638 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d3779-9835-415f-8221-86eb165ff2e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:12.434373 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d3779-9835-415f-8221-86eb165ff2e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:42:12.434736 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d3779-9835-415f-8221-86eb165ff2e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:12.437168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d3779-9835-415f-8221-86eb165ff2e0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:12.449857 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1375/2727] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:42:12.455117 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634be635-e26b-402f-93f9-4ba985fe2d08 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:12.464516 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634be635-e26b-402f-93f9-4ba985fe2d08 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:12.469403 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4830ac-2a05-4c6c-8e4f-039ba3564094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:12.470060 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4830ac-2a05-4c6c-8e4f-039ba3564094 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:12.470338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4830ac-2a05-4c6c-8e4f-039ba3564094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:12.470613 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4830ac-2a05-4c6c-8e4f-039ba3564094 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:12.471305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4830ac-2a05-4c6c-8e4f-039ba3564094 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:12.473210 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4830ac-2a05-4c6c-8e4f-039ba3564094 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:12.588028 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1353/2728] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:42:12 2022] POST /identity/v3/projects => generated 387 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:42:12.592260 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d3779-9835-415f-8221-86eb165ff2e0 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:12.602154 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d3779-9835-415f-8221-86eb165ff2e0 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:12.659690 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3038dda9-f17b-495b-b8a1-b19176d7decc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:12.660279 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038dda9-f17b-495b-b8a1-b19176d7decc admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:12.660610 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038dda9-f17b-495b-b8a1-b19176d7decc admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:12.660866 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038dda9-f17b-495b-b8a1-b19176d7decc admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:12.661556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3038dda9-f17b-495b-b8a1-b19176d7decc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:12.663560 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3038dda9-f17b-495b-b8a1-b19176d7decc admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:12.803024 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1376/2729] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:42:12 2022] POST /identity/v3/users => generated 294 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:42:12.807372 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4830ac-2a05-4c6c-8e4f-039ba3564094 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:12.818712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4830ac-2a05-4c6c-8e4f-039ba3564094 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:12.823431 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63556dcc-5006-4d0a-bc45-5b04ee9d5cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:12.824138 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63556dcc-5006-4d0a-bc45-5b04ee9d5cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:12.824433 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63556dcc-5006-4d0a-bc45-5b04ee9d5cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:12.824706 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63556dcc-5006-4d0a-bc45-5b04ee9d5cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:12.825619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63556dcc-5006-4d0a-bc45-5b04ee9d5cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:12.827236 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63556dcc-5006-4d0a-bc45-5b04ee9d5cf0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:12.837683 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1354/2730] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:42:12.841667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3038dda9-f17b-495b-b8a1-b19176d7decc admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:12.851823 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3038dda9-f17b-495b-b8a1-b19176d7decc admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:12.856564 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68220c3-330c-4094-ae48-4985377084f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:12.858000 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68220c3-330c-4094-ae48-4985377084f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:12.861634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68220c3-330c-4094-ae48-4985377084f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:12.861634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68220c3-330c-4094-ae48-4985377084f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:12.861634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68220c3-330c-4094-ae48-4985377084f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:12.861634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68220c3-330c-4094-ae48-4985377084f0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:12.871442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1377/2731] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:42:12.876462 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63556dcc-5006-4d0a-bc45-5b04ee9d5cf0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:12.886712 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63556dcc-5006-4d0a-bc45-5b04ee9d5cf0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:12.892099 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc1f5c-3250-4981-a375-44ea061f7b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:12.893021 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc1f5c-3250-4981-a375-44ea061f7b05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:12.893297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc1f5c-3250-4981-a375-44ea061f7b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:12.893572 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc1f5c-3250-4981-a375-44ea061f7b05 admin admin] PATH_INFO: `/v3/projects/5111713eb033408dae1e633bb8c371c4/users/c39fd10a5707480fa1f1a195ad658212/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:12.910869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc1f5c-3250-4981-a375-44ea061f7b05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5111713eb033408dae1e633bb8c371c4, user_id=c39fd10a5707480fa1f1a195ad658212, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:12.913501 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc1f5c-3250-4981-a375-44ea061f7b05 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:13.031655 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1355/2732] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:42:12 2022] PUT /identity/v3/projects/5111713eb033408dae1e633bb8c371c4/users/c39fd10a5707480fa1f1a195ad658212/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:13.043025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e71d1-5674-4e93-84aa-f2bf4ef44ab9 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:13.043318 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e71d1-5674-4e93-84aa-f2bf4ef44ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:13.043631 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e71d1-5674-4e93-84aa-f2bf4ef44ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:13.072246 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-8d3e71d1-5674-4e93-84aa-f2bf4ef44ab9 None None] MFA Rules not processed for user `c39fd10a5707480fa1f1a195ad658212`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:13.073451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-8d3e71d1-5674-4e93-84aa-f2bf4ef44ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:13.139418 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1378/2733] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:13.144070 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc1f5c-3250-4981-a375-44ea061f7b05 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:13.206280 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc1f5c-3250-4981-a375-44ea061f7b05 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:13.211076 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381afc2f-c17f-4865-a35e-c85366331ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:13.211774 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381afc2f-c17f-4865-a35e-c85366331ab5 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:13.212053 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381afc2f-c17f-4865-a35e-c85366331ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:13.212330 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381afc2f-c17f-4865-a35e-c85366331ab5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:13.213030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381afc2f-c17f-4865-a35e-c85366331ab5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:13.214924 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381afc2f-c17f-4865-a35e-c85366331ab5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:13.306797 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1356/2734] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:42:13 2022] POST /identity/v3/projects => generated 387 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:42:13.312169 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e71d1-5674-4e93-84aa-f2bf4ef44ab9 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:13.322585 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e71d1-5674-4e93-84aa-f2bf4ef44ab9 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:13.330851 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:13.381869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd5e93-8b03-4dc9-9325-f5267d5805ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:13.382511 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd5e93-8b03-4dc9-9325-f5267d5805ef admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:13.382796 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd5e93-8b03-4dc9-9325-f5267d5805ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:13.383079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd5e93-8b03-4dc9-9325-f5267d5805ef admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:13.383838 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd5e93-8b03-4dc9-9325-f5267d5805ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:13.385779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd5e93-8b03-4dc9-9325-f5267d5805ef admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:13.449581 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1379/2735] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:42:13 2022] POST /identity/v3/users => generated 300 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:42:13.454640 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381afc2f-c17f-4865-a35e-c85366331ab5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:13.465514 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381afc2f-c17f-4865-a35e-c85366331ab5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:13.470961 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92119056-214e-4e8e-a6a3-ed56d775aabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:13.471629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92119056-214e-4e8e-a6a3-ed56d775aabd admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:13.471917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92119056-214e-4e8e-a6a3-ed56d775aabd admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:13.472197 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92119056-214e-4e8e-a6a3-ed56d775aabd admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:13.473226 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92119056-214e-4e8e-a6a3-ed56d775aabd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:13.474552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92119056-214e-4e8e-a6a3-ed56d775aabd admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:13.485815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1357/2736] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:42:13.489980 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd5e93-8b03-4dc9-9325-f5267d5805ef admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:13.501166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd5e93-8b03-4dc9-9325-f5267d5805ef admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:13.505895 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31ac3f-7714-435d-bf5f-02819986b9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:13.506785 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31ac3f-7714-435d-bf5f-02819986b9c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:13.507068 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31ac3f-7714-435d-bf5f-02819986b9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:13.507346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce31ac3f-7714-435d-bf5f-02819986b9c9 admin admin] PATH_INFO: `/v3/projects/1e3970b5007b41e7837c42d63cf6c448/users/1990e5d9d88d4d20b7e87eec8e25cba4/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:13.523614 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31ac3f-7714-435d-bf5f-02819986b9c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e3970b5007b41e7837c42d63cf6c448, user_id=1990e5d9d88d4d20b7e87eec8e25cba4, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:13.526338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce31ac3f-7714-435d-bf5f-02819986b9c9 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:13.602747 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1380/2737] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:42:13 2022] PUT /identity/v3/projects/1e3970b5007b41e7837c42d63cf6c448/users/1990e5d9d88d4d20b7e87eec8e25cba4/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:13.607176 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92119056-214e-4e8e-a6a3-ed56d775aabd admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:13.618052 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92119056-214e-4e8e-a6a3-ed56d775aabd admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:13.677218 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa4678-a425-486d-8873-0a338533ed29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:13.677896 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa4678-a425-486d-8873-0a338533ed29 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:13.678183 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa4678-a425-486d-8873-0a338533ed29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:13.678464 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa4678-a425-486d-8873-0a338533ed29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:13.679277 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa4678-a425-486d-8873-0a338533ed29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:13.681041 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa4678-a425-486d-8873-0a338533ed29 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:13.683418 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:13.695669 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1358/2738] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:42:13.699825 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31ac3f-7714-435d-bf5f-02819986b9c9 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:13.711524 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce31ac3f-7714-435d-bf5f-02819986b9c9 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:13.716778 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95dbdce-4c41-4195-841f-d6b185070691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:13.718077 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dbdce-4c41-4195-841f-d6b185070691 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:13.718359 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dbdce-4c41-4195-841f-d6b185070691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:13.718632 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95dbdce-4c41-4195-841f-d6b185070691 admin admin] PATH_INFO: `/v3/projects/1e3970b5007b41e7837c42d63cf6c448/users/1990e5d9d88d4d20b7e87eec8e25cba4/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:13.720691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95dbdce-4c41-4195-841f-d6b185070691 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e3970b5007b41e7837c42d63cf6c448, user_id=1990e5d9d88d4d20b7e87eec8e25cba4, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:13.725929 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95dbdce-4c41-4195-841f-d6b185070691 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:13.734320 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1381/2739] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:42:13 2022] PUT /identity/v3/projects/1e3970b5007b41e7837c42d63cf6c448/users/1990e5d9d88d4d20b7e87eec8e25cba4/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:42:13.745468 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967a136-c402-4d68-9e25-f774e68efe76 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:13.745770 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967a136-c402-4d68-9e25-f774e68efe76 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:13.746228 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967a136-c402-4d68-9e25-f774e68efe76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:13.776192 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b967a136-c402-4d68-9e25-f774e68efe76 None None] MFA Rules not processed for user `1990e5d9d88d4d20b7e87eec8e25cba4`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:13.777428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b967a136-c402-4d68-9e25-f774e68efe76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:13.856839 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1359/2740] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:13.868548 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95dbdce-4c41-4195-841f-d6b185070691 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:13.870310 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a95dbdce-4c41-4195-841f-d6b185070691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:13.933001 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95dbdce-4c41-4195-841f-d6b185070691 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:13.937995 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cb29b8-ccee-4481-8587-2578bb175fc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:13.938676 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb29b8-ccee-4481-8587-2578bb175fc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:13.938955 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb29b8-ccee-4481-8587-2578bb175fc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:13.939231 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb29b8-ccee-4481-8587-2578bb175fc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:13.949489 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cb29b8-ccee-4481-8587-2578bb175fc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:13.952288 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cb29b8-ccee-4481-8587-2578bb175fc3 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:13.962702 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1382/2741] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:42:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:42:14.107391 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967a136-c402-4d68-9e25-f774e68efe76 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:14.171634 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967a136-c402-4d68-9e25-f774e68efe76 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:14.176291 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefcc0c-7c9b-4c45-8ef4-02bda81ee99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:14.177153 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefcc0c-7c9b-4c45-8ef4-02bda81ee99d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:14.177436 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefcc0c-7c9b-4c45-8ef4-02bda81ee99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:14.177708 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefcc0c-7c9b-4c45-8ef4-02bda81ee99d admin admin] PATH_INFO: `/v3/users/c39fd10a5707480fa1f1a195ad658212` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:14.178909 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefcc0c-7c9b-4c45-8ef4-02bda81ee99d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c39fd10a5707480fa1f1a195ad658212)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:14.180883 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefcc0c-7c9b-4c45-8ef4-02bda81ee99d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:14.399501 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dcefcc0c-7c9b-4c45-8ef4-02bda81ee99d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c39fd10a5707480fa1f1a195ad658212'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:14.400191 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dcefcc0c-7c9b-4c45-8ef4-02bda81ee99d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c39fd10a5707480fa1f1a195ad658212'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:14.476809 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dcefcc0c-7c9b-4c45-8ef4-02bda81ee99d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c39fd10a5707480fa1f1a195ad658212'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:14.485997 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dcefcc0c-7c9b-4c45-8ef4-02bda81ee99d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c39fd10a5707480fa1f1a195ad658212'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:14.486704 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-dcefcc0c-7c9b-4c45-8ef4-02bda81ee99d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c39fd10a5707480fa1f1a195ad658212'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:14.500940 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:14.505251 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1360/2742] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:42:14 2022] DELETE /identity/v3/users/c39fd10a5707480fa1f1a195ad658212 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:14.509890 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7758c-68f0-4309-a9d8-d739f3eb137d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:14.510367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7758c-68f0-4309-a9d8-d739f3eb137d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:14.510487 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7758c-68f0-4309-a9d8-d739f3eb137d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:14.529013 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-1bf7758c-68f0-4309-a9d8-d739f3eb137d None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:14.530170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1bf7758c-68f0-4309-a9d8-d739f3eb137d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:14.595873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1383/2743] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:42:14.598314 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefcc0c-7c9b-4c45-8ef4-02bda81ee99d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:14.600352 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dcefcc0c-7c9b-4c45-8ef4-02bda81ee99d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:14.608455 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7758c-68f0-4309-a9d8-d739f3eb137d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:14.609948 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-1bf7758c-68f0-4309-a9d8-d739f3eb137d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:14.614465 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefcc0c-7c9b-4c45-8ef4-02bda81ee99d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:14.619637 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8be618-98d7-4a71-adcd-989dfe0e4cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:14.620100 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:14.620678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8be618-98d7-4a71-adcd-989dfe0e4cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:14.620968 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8be618-98d7-4a71-adcd-989dfe0e4cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:14.621248 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8be618-98d7-4a71-adcd-989dfe0e4cab admin admin] PATH_INFO: `/v3/users/3579fb153d3340c282ddffab5d3c6203` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:14.622659 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8be618-98d7-4a71-adcd-989dfe0e4cab admin admin] RBAC: Authorizing `identity:delete_user(user_id=3579fb153d3340c282ddffab5d3c6203)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:14.624868 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8be618-98d7-4a71-adcd-989dfe0e4cab admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:14.673463 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7758c-68f0-4309-a9d8-d739f3eb137d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:14.677211 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac96214-dc9d-492b-9827-69a9861d152f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:14.677852 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac96214-dc9d-492b-9827-69a9861d152f service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:14.678137 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac96214-dc9d-492b-9827-69a9861d152f service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:14.678417 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac96214-dc9d-492b-9827-69a9861d152f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:14.688112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac96214-dc9d-492b-9827-69a9861d152f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:14.689558 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac96214-dc9d-492b-9827-69a9861d152f service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:14.699630 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1384/2744] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:42:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:42:14.830071 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ec8be618-98d7-4a71-adcd-989dfe0e4cab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3579fb153d3340c282ddffab5d3c6203'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:14.830742 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ec8be618-98d7-4a71-adcd-989dfe0e4cab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3579fb153d3340c282ddffab5d3c6203'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:14.958578 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ec8be618-98d7-4a71-adcd-989dfe0e4cab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3579fb153d3340c282ddffab5d3c6203'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:14.969713 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ec8be618-98d7-4a71-adcd-989dfe0e4cab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3579fb153d3340c282ddffab5d3c6203'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:14.971085 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ec8be618-98d7-4a71-adcd-989dfe0e4cab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3579fb153d3340c282ddffab5d3c6203'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:15.001484 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1361/2745] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:42:14 2022] DELETE /identity/v3/users/3579fb153d3340c282ddffab5d3c6203 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:42:15.005187 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2404f20-aea5-48f9-b3b8-6f9c13e9ce7b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:15.005479 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2404f20-aea5-48f9-b3b8-6f9c13e9ce7b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:15.005777 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2404f20-aea5-48f9-b3b8-6f9c13e9ce7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:15.028475 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-d2404f20-aea5-48f9-b3b8-6f9c13e9ce7b None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:15.030681 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d2404f20-aea5-48f9-b3b8-6f9c13e9ce7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:15.100700 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1385/2746] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:15.112944 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8be618-98d7-4a71-adcd-989dfe0e4cab admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:15.114732 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-ec8be618-98d7-4a71-adcd-989dfe0e4cab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:15.174691 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8be618-98d7-4a71-adcd-989dfe0e4cab admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:15.179203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c774633-8c1c-46e2-938a-b48ad3216fe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:15.179863 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c774633-8c1c-46e2-938a-b48ad3216fe4 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:15.180160 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c774633-8c1c-46e2-938a-b48ad3216fe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:15.180438 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c774633-8c1c-46e2-938a-b48ad3216fe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:15.190912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c774633-8c1c-46e2-938a-b48ad3216fe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:15.192624 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c774633-8c1c-46e2-938a-b48ad3216fe4 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:15.202424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1362/2747] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:42:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:42:15.218442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2404f20-aea5-48f9-b3b8-6f9c13e9ce7b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:15.219883 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d2404f20-aea5-48f9-b3b8-6f9c13e9ce7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:15.233085 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2404f20-aea5-48f9-b3b8-6f9c13e9ce7b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:15.238450 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568c215-712e-46b5-95f5-4c392728be54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:15.239404 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568c215-712e-46b5-95f5-4c392728be54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:15.239725 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568c215-712e-46b5-95f5-4c392728be54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:15.240007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568c215-712e-46b5-95f5-4c392728be54 admin admin] PATH_INFO: `/v3/projects/5111713eb033408dae1e633bb8c371c4` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:15.241237 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4568c215-712e-46b5-95f5-4c392728be54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5111713eb033408dae1e633bb8c371c4)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:15.243298 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4568c215-712e-46b5-95f5-4c392728be54 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:15.358399 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4568c215-712e-46b5-95f5-4c392728be54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5111713eb033408dae1e633bb8c371c4', 'request_id': 'req-4568c215-712e-46b5-95f5-4c392728be54'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:15.444315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4568c215-712e-46b5-95f5-4c392728be54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5111713eb033408dae1e633bb8c371c4', 'request_id': 'req-4568c215-712e-46b5-95f5-4c392728be54'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:15.452564 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4568c215-712e-46b5-95f5-4c392728be54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5111713eb033408dae1e633bb8c371c4', 'request_id': 'req-4568c215-712e-46b5-95f5-4c392728be54'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:15.466822 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4568c215-712e-46b5-95f5-4c392728be54 admin admin] The token cache is being invalidate because project 5111713eb033408dae1e633bb8c371c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:42:15.467833 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4568c215-712e-46b5-95f5-4c392728be54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:15.468555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-4568c215-712e-46b5-95f5-4c392728be54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:15.470190 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1386/2748] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:42:15 2022] DELETE /identity/v3/projects/5111713eb033408dae1e633bb8c371c4 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:15.474188 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c774633-8c1c-46e2-938a-b48ad3216fe4 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:15.488070 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c774633-8c1c-46e2-938a-b48ad3216fe4 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:15.547590 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c16cc-bbde-4de0-ab81-663005d9b5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:15.548465 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c16cc-bbde-4de0-ab81-663005d9b5f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:15.548752 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c16cc-bbde-4de0-ab81-663005d9b5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:15.549030 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c16cc-bbde-4de0-ab81-663005d9b5f0 admin admin] PATH_INFO: `/v3/users/1990e5d9d88d4d20b7e87eec8e25cba4` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:15.550305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c16cc-bbde-4de0-ab81-663005d9b5f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1990e5d9d88d4d20b7e87eec8e25cba4)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:15.552334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c16cc-bbde-4de0-ab81-663005d9b5f0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:15.570115 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:15.731597 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568c215-712e-46b5-95f5-4c392728be54 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:15.733407 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-4568c215-712e-46b5-95f5-4c392728be54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:15.747285 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568c215-712e-46b5-95f5-4c392728be54 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:15.752918 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89624c-0b43-4eef-ac77-a1b56e7b8f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:15.753801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89624c-0b43-4eef-ac77-a1b56e7b8f99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:15.754082 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89624c-0b43-4eef-ac77-a1b56e7b8f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:15.754759 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89624c-0b43-4eef-ac77-a1b56e7b8f99 admin admin] PATH_INFO: `/v3/projects/450b26a225464051b4c488c847ae87f5` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:15.755644 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89624c-0b43-4eef-ac77-a1b56e7b8f99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=450b26a225464051b4c488c847ae87f5)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:15.757628 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89624c-0b43-4eef-ac77-a1b56e7b8f99 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:15.795274 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-014c16cc-bbde-4de0-ab81-663005d9b5f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1990e5d9d88d4d20b7e87eec8e25cba4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:15.795967 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-014c16cc-bbde-4de0-ab81-663005d9b5f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1990e5d9d88d4d20b7e87eec8e25cba4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:15.869766 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:15.884410 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-014c16cc-bbde-4de0-ab81-663005d9b5f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1990e5d9d88d4d20b7e87eec8e25cba4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:15.884635 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0b89624c-0b43-4eef-ac77-a1b56e7b8f99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '450b26a225464051b4c488c847ae87f5', 'request_id': 'req-0b89624c-0b43-4eef-ac77-a1b56e7b8f99'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:15.892523 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-014c16cc-bbde-4de0-ab81-663005d9b5f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1990e5d9d88d4d20b7e87eec8e25cba4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:15.893199 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-014c16cc-bbde-4de0-ab81-663005d9b5f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1990e5d9d88d4d20b7e87eec8e25cba4'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:15.910765 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1363/2749] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:42:15 2022] DELETE /identity/v3/users/1990e5d9d88d4d20b7e87eec8e25cba4 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:15.968156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0b89624c-0b43-4eef-ac77-a1b56e7b8f99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '450b26a225464051b4c488c847ae87f5', 'request_id': 'req-0b89624c-0b43-4eef-ac77-a1b56e7b8f99'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:15.975546 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0b89624c-0b43-4eef-ac77-a1b56e7b8f99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '450b26a225464051b4c488c847ae87f5', 'request_id': 'req-0b89624c-0b43-4eef-ac77-a1b56e7b8f99'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:15.988474 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0b89624c-0b43-4eef-ac77-a1b56e7b8f99 admin admin] The token cache is being invalidate because project 450b26a225464051b4c488c847ae87f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:42:15.991354 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0b89624c-0b43-4eef-ac77-a1b56e7b8f99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:15.991354 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0b89624c-0b43-4eef-ac77-a1b56e7b8f99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:15.991354 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1387/2750] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:42:15 2022] DELETE /identity/v3/projects/450b26a225464051b4c488c847ae87f5 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:42:15.999802 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c16cc-bbde-4de0-ab81-663005d9b5f0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:16.001512 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-014c16cc-bbde-4de0-ab81-663005d9b5f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:16.072472 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c16cc-bbde-4de0-ab81-663005d9b5f0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:16.077853 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee559b94-1592-44bb-90dd-c02d03d1bc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:16.078743 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee559b94-1592-44bb-90dd-c02d03d1bc51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:16.079026 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee559b94-1592-44bb-90dd-c02d03d1bc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:16.079302 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee559b94-1592-44bb-90dd-c02d03d1bc51 admin admin] PATH_INFO: `/v3/users/3818552a1d1347e8a767c93e22e49d8a` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:16.080546 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee559b94-1592-44bb-90dd-c02d03d1bc51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3818552a1d1347e8a767c93e22e49d8a)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:16.082516 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee559b94-1592-44bb-90dd-c02d03d1bc51 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:16.238049 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ee559b94-1592-44bb-90dd-c02d03d1bc51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3818552a1d1347e8a767c93e22e49d8a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:16.238735 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ee559b94-1592-44bb-90dd-c02d03d1bc51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3818552a1d1347e8a767c93e22e49d8a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:16.315015 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89624c-0b43-4eef-ac77-a1b56e7b8f99 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:16.316586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-0b89624c-0b43-4eef-ac77-a1b56e7b8f99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:16.334499 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ee559b94-1592-44bb-90dd-c02d03d1bc51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3818552a1d1347e8a767c93e22e49d8a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:16.364540 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ee559b94-1592-44bb-90dd-c02d03d1bc51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3818552a1d1347e8a767c93e22e49d8a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:16.365679 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-ee559b94-1592-44bb-90dd-c02d03d1bc51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3818552a1d1347e8a767c93e22e49d8a'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:16.386987 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1364/2751] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:42:15 2022] DELETE /identity/v3/users/3818552a1d1347e8a767c93e22e49d8a => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:42:16.393666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89624c-0b43-4eef-ac77-a1b56e7b8f99 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:16.465667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090cbdc-a516-4431-bdab-c29277641276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:16.467973 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090cbdc-a516-4431-bdab-c29277641276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:16.468271 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090cbdc-a516-4431-bdab-c29277641276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:16.468554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090cbdc-a516-4431-bdab-c29277641276 admin admin] PATH_INFO: `/v3/projects/1e3970b5007b41e7837c42d63cf6c448` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:16.469978 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090cbdc-a516-4431-bdab-c29277641276 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e3970b5007b41e7837c42d63cf6c448)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:16.473079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090cbdc-a516-4431-bdab-c29277641276 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:16.666809 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5090cbdc-a516-4431-bdab-c29277641276 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e3970b5007b41e7837c42d63cf6c448', 'request_id': 'req-5090cbdc-a516-4431-bdab-c29277641276'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:16.764064 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5090cbdc-a516-4431-bdab-c29277641276 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e3970b5007b41e7837c42d63cf6c448', 'request_id': 'req-5090cbdc-a516-4431-bdab-c29277641276'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:16.771155 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5090cbdc-a516-4431-bdab-c29277641276 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e3970b5007b41e7837c42d63cf6c448', 'request_id': 'req-5090cbdc-a516-4431-bdab-c29277641276'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:16.775492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:16.783032 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5090cbdc-a516-4431-bdab-c29277641276 admin admin] The token cache is being invalidate because project 1e3970b5007b41e7837c42d63cf6c448 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:42:16.783338 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5090cbdc-a516-4431-bdab-c29277641276 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:16.784091 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-5090cbdc-a516-4431-bdab-c29277641276 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:16.785664 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1388/2752] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:42:16 2022] DELETE /identity/v3/projects/1e3970b5007b41e7837c42d63cf6c448 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:42:16.792390 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44124e57-b671-4b15-98dd-d85a0b97b743 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:16.792788 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44124e57-b671-4b15-98dd-d85a0b97b743 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:16.793142 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44124e57-b671-4b15-98dd-d85a0b97b743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:16.813589 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-44124e57-b671-4b15-98dd-d85a0b97b743 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:16.814970 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-44124e57-b671-4b15-98dd-d85a0b97b743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:16.824040 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:16.877986 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1365/2753] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:16.886096 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d2929-eb11-41c3-97f5-18ad26e4ce1e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:16.886386 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d2929-eb11-41c3-97f5-18ad26e4ce1e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:16.886669 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d2929-eb11-41c3-97f5-18ad26e4ce1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:16.906016 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-861d2929-eb11-41c3-97f5-18ad26e4ce1e None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:16.907257 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-861d2929-eb11-41c3-97f5-18ad26e4ce1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:16.916285 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1389/2754] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:16.920841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44124e57-b671-4b15-98dd-d85a0b97b743 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:16.930489 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d2929-eb11-41c3-97f5-18ad26e4ce1e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:16.937287 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44124e57-b671-4b15-98dd-d85a0b97b743 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:16.937436 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-861d2929-eb11-41c3-97f5-18ad26e4ce1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:16.937560 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6131a48-ba23-4f19-be5f-258873d1217e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:16.937560 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6131a48-ba23-4f19-be5f-258873d1217e admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:16.937560 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6131a48-ba23-4f19-be5f-258873d1217e admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:16.937560 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6131a48-ba23-4f19-be5f-258873d1217e admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:16.937920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6131a48-ba23-4f19-be5f-258873d1217e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:42:16.938160 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6131a48-ba23-4f19-be5f-258873d1217e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:16.939880 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6131a48-ba23-4f19-be5f-258873d1217e admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:16.948283 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d2929-eb11-41c3-97f5-18ad26e4ce1e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:16.953361 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123bbab-e7ca-48a1-8c43-be68164191e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:16.954357 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123bbab-e7ca-48a1-8c43-be68164191e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:16.955127 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123bbab-e7ca-48a1-8c43-be68164191e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:16.955615 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123bbab-e7ca-48a1-8c43-be68164191e5 admin admin] PATH_INFO: `/v3/projects/2a9c1b34b4074e0081684b2dc1e98182` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:16.957086 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3123bbab-e7ca-48a1-8c43-be68164191e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a9c1b34b4074e0081684b2dc1e98182)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:16.959210 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3123bbab-e7ca-48a1-8c43-be68164191e5 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:16.961893 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1366/2755] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:42:16.968044 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6131a48-ba23-4f19-be5f-258873d1217e admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:16.984384 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6131a48-ba23-4f19-be5f-258873d1217e admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:16.989600 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea952c7b-1eea-47d5-b0f7-505b0e5962f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:16.990327 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea952c7b-1eea-47d5-b0f7-505b0e5962f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:16.990609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea952c7b-1eea-47d5-b0f7-505b0e5962f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:16.990890 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea952c7b-1eea-47d5-b0f7-505b0e5962f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:16.991617 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea952c7b-1eea-47d5-b0f7-505b0e5962f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:16.993552 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea952c7b-1eea-47d5-b0f7-505b0e5962f0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:17.106900 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1367/2756] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:42:16 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:42:17.111590 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea952c7b-1eea-47d5-b0f7-505b0e5962f0 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:17.117965 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3123bbab-e7ca-48a1-8c43-be68164191e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a9c1b34b4074e0081684b2dc1e98182', 'request_id': 'req-3123bbab-e7ca-48a1-8c43-be68164191e5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:17.175405 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea952c7b-1eea-47d5-b0f7-505b0e5962f0 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:17.180493 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f79249-9af6-4e2d-99fd-cfb1ec483db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:17.181167 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f79249-9af6-4e2d-99fd-cfb1ec483db9 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:17.181481 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f79249-9af6-4e2d-99fd-cfb1ec483db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:17.181763 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f79249-9af6-4e2d-99fd-cfb1ec483db9 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:17.182480 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f79249-9af6-4e2d-99fd-cfb1ec483db9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:17.184534 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f79249-9af6-4e2d-99fd-cfb1ec483db9 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:17.192442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3123bbab-e7ca-48a1-8c43-be68164191e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a9c1b34b4074e0081684b2dc1e98182', 'request_id': 'req-3123bbab-e7ca-48a1-8c43-be68164191e5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:17.200252 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3123bbab-e7ca-48a1-8c43-be68164191e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a9c1b34b4074e0081684b2dc1e98182', 'request_id': 'req-3123bbab-e7ca-48a1-8c43-be68164191e5'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:17.212540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3123bbab-e7ca-48a1-8c43-be68164191e5 admin admin] The token cache is being invalidate because project 2a9c1b34b4074e0081684b2dc1e98182 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:42:17.212829 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3123bbab-e7ca-48a1-8c43-be68164191e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:17.213497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3123bbab-e7ca-48a1-8c43-be68164191e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:17.215086 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1390/2757] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:42:16 2022] DELETE /identity/v3/projects/2a9c1b34b4074e0081684b2dc1e98182 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:42:17.220263 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123bbab-e7ca-48a1-8c43-be68164191e5 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:17.221605 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3123bbab-e7ca-48a1-8c43-be68164191e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:17.286931 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123bbab-e7ca-48a1-8c43-be68164191e5 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:17.291643 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54893b11-fde1-4f7b-8239-19490bca7c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:17.292555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54893b11-fde1-4f7b-8239-19490bca7c08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:17.292841 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54893b11-fde1-4f7b-8239-19490bca7c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:17.293118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54893b11-fde1-4f7b-8239-19490bca7c08 admin admin] PATH_INFO: `/v3/users/123f35fed9534b7f8d4441bc65bcb2ef` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:17.294327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54893b11-fde1-4f7b-8239-19490bca7c08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=123f35fed9534b7f8d4441bc65bcb2ef)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:17.296323 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54893b11-fde1-4f7b-8239-19490bca7c08 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:17.319011 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1368/2758] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:42:17 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:42:17.323122 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f79249-9af6-4e2d-99fd-cfb1ec483db9 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:17.324767 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f0f79249-9af6-4e2d-99fd-cfb1ec483db9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:17.337815 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f79249-9af6-4e2d-99fd-cfb1ec483db9 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:17.343698 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4f76b-e5d1-4485-b420-98dbc26ef53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:17.344346 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4f76b-e5d1-4485-b420-98dbc26ef53b admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:17.344627 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4f76b-e5d1-4485-b420-98dbc26ef53b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:17.344909 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4f76b-e5d1-4485-b420-98dbc26ef53b admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:17.345610 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea4f76b-e5d1-4485-b420-98dbc26ef53b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:17.347194 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea4f76b-e5d1-4485-b420-98dbc26ef53b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:17.357037 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1369/2759] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:42:17.361109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4f76b-e5d1-4485-b420-98dbc26ef53b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:17.370594 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4f76b-e5d1-4485-b420-98dbc26ef53b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:17.375609 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51f029-6a93-47b6-ac17-03a97d28455f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:17.376239 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51f029-6a93-47b6-ac17-03a97d28455f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:17.376526 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51f029-6a93-47b6-ac17-03a97d28455f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:17.376798 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51f029-6a93-47b6-ac17-03a97d28455f admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:17.377498 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e51f029-6a93-47b6-ac17-03a97d28455f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:17.379069 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e51f029-6a93-47b6-ac17-03a97d28455f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:17.390615 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1370/2760] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:42:17.394617 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51f029-6a93-47b6-ac17-03a97d28455f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:17.404728 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51f029-6a93-47b6-ac17-03a97d28455f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:17.409835 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e128d4-c2d2-4475-a4d0-662c73bf7718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:17.410697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e128d4-c2d2-4475-a4d0-662c73bf7718 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:17.410973 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e128d4-c2d2-4475-a4d0-662c73bf7718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:17.413290 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e128d4-c2d2-4475-a4d0-662c73bf7718 admin admin] PATH_INFO: `/v3/projects/79ea8306f28d4c838e5b82645df11a21/users/38fedc0d90b54e44978fc79b5a532ca0/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:17.425808 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e128d4-c2d2-4475-a4d0-662c73bf7718 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79ea8306f28d4c838e5b82645df11a21, user_id=38fedc0d90b54e44978fc79b5a532ca0, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:17.428475 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e128d4-c2d2-4475-a4d0-662c73bf7718 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:17.568920 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1371/2761] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:42:17 2022] PUT /identity/v3/projects/79ea8306f28d4c838e5b82645df11a21/users/38fedc0d90b54e44978fc79b5a532ca0/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:42:17.579996 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06130bba-3c4c-48da-80d1-40d7d9526144 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:17.580212 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06130bba-3c4c-48da-80d1-40d7d9526144 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:17.580512 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06130bba-3c4c-48da-80d1-40d7d9526144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:17.601923 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-54893b11-fde1-4f7b-8239-19490bca7c08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '123f35fed9534b7f8d4441bc65bcb2ef'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:17.602726 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-54893b11-fde1-4f7b-8239-19490bca7c08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '123f35fed9534b7f8d4441bc65bcb2ef'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:17.608480 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-06130bba-3c4c-48da-80d1-40d7d9526144 None None] MFA Rules not processed for user `38fedc0d90b54e44978fc79b5a532ca0`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:17.609737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-06130bba-3c4c-48da-80d1-40d7d9526144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:17.610365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-54893b11-fde1-4f7b-8239-19490bca7c08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '123f35fed9534b7f8d4441bc65bcb2ef'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:17.618810 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:17.661151 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-54893b11-fde1-4f7b-8239-19490bca7c08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '123f35fed9534b7f8d4441bc65bcb2ef'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:17.662055 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-54893b11-fde1-4f7b-8239-19490bca7c08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '123f35fed9534b7f8d4441bc65bcb2ef'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:17.670632 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:17.675673 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1372/2762] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:42:17.687803 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1391/2763] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:42:17 2022] DELETE /identity/v3/users/123f35fed9534b7f8d4441bc65bcb2ef => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:42:17.688588 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06130bba-3c4c-48da-80d1-40d7d9526144 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:17.690057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-06130bba-3c4c-48da-80d1-40d7d9526144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:17.754917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06130bba-3c4c-48da-80d1-40d7d9526144 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:17.803877 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f8f4c9-3a0d-4cf7-87a0-70084d2437b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:17.804571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8f4c9-3a0d-4cf7-87a0-70084d2437b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:17.804860 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8f4c9-3a0d-4cf7-87a0-70084d2437b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:17.805140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f8f4c9-3a0d-4cf7-87a0-70084d2437b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:17.807023 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f1f8f4c9-3a0d-4cf7-87a0-70084d2437b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:17.871398 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f8f4c9-3a0d-4cf7-87a0-70084d2437b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:17.871398 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f8f4c9-3a0d-4cf7-87a0-70084d2437b8 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:18.025465 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1373/2764] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:42:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:42:18.331159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54893b11-fde1-4f7b-8239-19490bca7c08 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:18.336632 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-54893b11-fde1-4f7b-8239-19490bca7c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:18.419312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54893b11-fde1-4f7b-8239-19490bca7c08 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:18.425178 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06da4d-2767-4afa-80ed-eb3a7ad73dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:18.426041 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06da4d-2767-4afa-80ed-eb3a7ad73dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:18.426321 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06da4d-2767-4afa-80ed-eb3a7ad73dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:18.426595 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06da4d-2767-4afa-80ed-eb3a7ad73dc2 admin admin] PATH_INFO: `/v3/projects/11e37a9637df45ecbba5343b6022407d` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:18.427799 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06da4d-2767-4afa-80ed-eb3a7ad73dc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11e37a9637df45ecbba5343b6022407d)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:18.429783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06da4d-2767-4afa-80ed-eb3a7ad73dc2 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:18.535101 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0b06da4d-2767-4afa-80ed-eb3a7ad73dc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11e37a9637df45ecbba5343b6022407d', 'request_id': 'req-0b06da4d-2767-4afa-80ed-eb3a7ad73dc2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:18.609417 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0b06da4d-2767-4afa-80ed-eb3a7ad73dc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11e37a9637df45ecbba5343b6022407d', 'request_id': 'req-0b06da4d-2767-4afa-80ed-eb3a7ad73dc2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:18.616061 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0b06da4d-2767-4afa-80ed-eb3a7ad73dc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11e37a9637df45ecbba5343b6022407d', 'request_id': 'req-0b06da4d-2767-4afa-80ed-eb3a7ad73dc2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:18.627640 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0b06da4d-2767-4afa-80ed-eb3a7ad73dc2 admin admin] The token cache is being invalidate because project 11e37a9637df45ecbba5343b6022407d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:42:18.627934 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0b06da4d-2767-4afa-80ed-eb3a7ad73dc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:18.628587 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-0b06da4d-2767-4afa-80ed-eb3a7ad73dc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:18.630139 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1392/2765] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:42:18 2022] DELETE /identity/v3/projects/11e37a9637df45ecbba5343b6022407d => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:18.635080 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d257b4b-8367-4f36-9903-f1d10adbf47a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:18.635588 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d257b4b-8367-4f36-9903-f1d10adbf47a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:18.635718 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d257b4b-8367-4f36-9903-f1d10adbf47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:18.657182 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-7d257b4b-8367-4f36-9903-f1d10adbf47a None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:18.658418 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-7d257b4b-8367-4f36-9903-f1d10adbf47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:18.678575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:18.725494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1374/2766] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:18.791873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4dd9a-40e7-4e0b-83fe-c9b5a8496b32 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:18.792168 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4dd9a-40e7-4e0b-83fe-c9b5a8496b32 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:18.792476 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4dd9a-40e7-4e0b-83fe-c9b5a8496b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:18.812809 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-6ef4dd9a-40e7-4e0b-83fe-c9b5a8496b32 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:18.814041 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-6ef4dd9a-40e7-4e0b-83fe-c9b5a8496b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:18.823519 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1393/2767] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:18.828539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d257b4b-8367-4f36-9903-f1d10adbf47a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:18.837534 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d257b4b-8367-4f36-9903-f1d10adbf47a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:18.844442 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b109a0a-c27e-4a08-b55d-0ee19451e91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:18.845137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b109a0a-c27e-4a08-b55d-0ee19451e91f admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:18.845417 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b109a0a-c27e-4a08-b55d-0ee19451e91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:18.845692 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b109a0a-c27e-4a08-b55d-0ee19451e91f admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:18.846690 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b109a0a-c27e-4a08-b55d-0ee19451e91f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:42:18.847153 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b109a0a-c27e-4a08-b55d-0ee19451e91f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:18.848813 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b109a0a-c27e-4a08-b55d-0ee19451e91f admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:18.861818 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1375/2768] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:42:18.866226 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef4dd9a-40e7-4e0b-83fe-c9b5a8496b32 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:18.876672 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef4dd9a-40e7-4e0b-83fe-c9b5a8496b32 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:18.881540 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3ef5c-7114-4470-9d82-f5c33d617a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:18.882200 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3ef5c-7114-4470-9d82-f5c33d617a44 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:18.882485 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3ef5c-7114-4470-9d82-f5c33d617a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:18.882764 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3ef5c-7114-4470-9d82-f5c33d617a44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:18.883486 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3ef5c-7114-4470-9d82-f5c33d617a44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:18.885459 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3ef5c-7114-4470-9d82-f5c33d617a44 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:19.023264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1394/2769] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:42:18 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:42:19.027745 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b109a0a-c27e-4a08-b55d-0ee19451e91f admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:19.038183 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b109a0a-c27e-4a08-b55d-0ee19451e91f admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:19.098384 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09bde76-36f1-48cb-a804-8ae69634009b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:19.099054 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09bde76-36f1-48cb-a804-8ae69634009b admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:19.099340 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09bde76-36f1-48cb-a804-8ae69634009b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:19.099654 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09bde76-36f1-48cb-a804-8ae69634009b admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:19.100366 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09bde76-36f1-48cb-a804-8ae69634009b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:19.102314 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09bde76-36f1-48cb-a804-8ae69634009b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:19.197370 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1376/2770] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:42:19 2022] POST /identity/v3/users => generated 287 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:42:19.201600 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3ef5c-7114-4470-9d82-f5c33d617a44 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:19.214666 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3ef5c-7114-4470-9d82-f5c33d617a44 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:19.220521 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4accb50e-6b32-49a1-81c0-9d7cdaafcbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:19.221332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4accb50e-6b32-49a1-81c0-9d7cdaafcbbc admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:19.221690 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4accb50e-6b32-49a1-81c0-9d7cdaafcbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:19.222040 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4accb50e-6b32-49a1-81c0-9d7cdaafcbbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:19.222843 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4accb50e-6b32-49a1-81c0-9d7cdaafcbbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:19.224578 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4accb50e-6b32-49a1-81c0-9d7cdaafcbbc admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:19.237220 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1395/2771] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:42:19.241467 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09bde76-36f1-48cb-a804-8ae69634009b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:19.252446 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09bde76-36f1-48cb-a804-8ae69634009b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:19.257084 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2545a94f-5f0e-421c-b3d3-5c40ce3897d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:19.257746 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545a94f-5f0e-421c-b3d3-5c40ce3897d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:19.258027 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545a94f-5f0e-421c-b3d3-5c40ce3897d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:19.258299 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2545a94f-5f0e-421c-b3d3-5c40ce3897d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:19.259019 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545a94f-5f0e-421c-b3d3-5c40ce3897d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:19.260632 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2545a94f-5f0e-421c-b3d3-5c40ce3897d8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:19.271308 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1377/2772] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:42:19.275867 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4accb50e-6b32-49a1-81c0-9d7cdaafcbbc admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:19.286847 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4accb50e-6b32-49a1-81c0-9d7cdaafcbbc admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:19.291656 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb1b98-9ac9-4f61-828f-9bcc242d9529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:19.292583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb1b98-9ac9-4f61-828f-9bcc242d9529 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:19.292868 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb1b98-9ac9-4f61-828f-9bcc242d9529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:19.293145 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb1b98-9ac9-4f61-828f-9bcc242d9529 admin admin] PATH_INFO: `/v3/projects/fc48c4912c8647d482593a28474d6b14/users/f58941916ebe4a68a9a9a2b04b745ee2/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:19.308423 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb1b98-9ac9-4f61-828f-9bcc242d9529 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc48c4912c8647d482593a28474d6b14, user_id=f58941916ebe4a68a9a9a2b04b745ee2, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:19.311117 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb1b98-9ac9-4f61-828f-9bcc242d9529 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:19.397080 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1396/2773] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:42:19 2022] PUT /identity/v3/projects/fc48c4912c8647d482593a28474d6b14/users/f58941916ebe4a68a9a9a2b04b745ee2/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:19.402405 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81c640-1a76-46b4-998c-3c34d748a72c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:19.402718 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81c640-1a76-46b4-998c-3c34d748a72c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:19.403022 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81c640-1a76-46b4-998c-3c34d748a72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:19.423689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-5c81c640-1a76-46b4-998c-3c34d748a72c None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:19.424941 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5c81c640-1a76-46b4-998c-3c34d748a72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:19.499240 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1378/2774] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:19.510124 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb1b98-9ac9-4f61-828f-9bcc242d9529 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:19.511910 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cceb1b98-9ac9-4f61-828f-9bcc242d9529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:19.532403 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:19.571384 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb1b98-9ac9-4f61-828f-9bcc242d9529 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:19.575069 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd12656-3c21-4657-a0c8-1bdb8d0c217b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:19.575746 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd12656-3c21-4657-a0c8-1bdb8d0c217b service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:19.576033 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd12656-3c21-4657-a0c8-1bdb8d0c217b service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:19.576312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd12656-3c21-4657-a0c8-1bdb8d0c217b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:19.586975 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd12656-3c21-4657-a0c8-1bdb8d0c217b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:19.588492 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd12656-3c21-4657-a0c8-1bdb8d0c217b service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:19.599296 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1397/2775] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:42:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:42:20.573262 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81c640-1a76-46b4-998c-3c34d748a72c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:20.574965 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5c81c640-1a76-46b4-998c-3c34d748a72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:20.588111 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81c640-1a76-46b4-998c-3c34d748a72c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:20.593708 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7d615-5195-4bb5-b7a0-613d3aed474d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:20.594592 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7d615-5195-4bb5-b7a0-613d3aed474d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:20.594875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7d615-5195-4bb5-b7a0-613d3aed474d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:20.595148 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7d615-5195-4bb5-b7a0-613d3aed474d admin admin] PATH_INFO: `/v3/users/528fc09c1cde4b50ab91560be809d9a6` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:20.596408 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7d615-5195-4bb5-b7a0-613d3aed474d admin admin] RBAC: Authorizing `identity:delete_user(user_id=528fc09c1cde4b50ab91560be809d9a6)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:20.598412 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7d615-5195-4bb5-b7a0-613d3aed474d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:20.622881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:20.791808 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d2e7d615-5195-4bb5-b7a0-613d3aed474d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '528fc09c1cde4b50ab91560be809d9a6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:20.792483 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d2e7d615-5195-4bb5-b7a0-613d3aed474d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '528fc09c1cde4b50ab91560be809d9a6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:20.881128 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d2e7d615-5195-4bb5-b7a0-613d3aed474d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '528fc09c1cde4b50ab91560be809d9a6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:20.889211 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d2e7d615-5195-4bb5-b7a0-613d3aed474d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '528fc09c1cde4b50ab91560be809d9a6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:20.890045 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-d2e7d615-5195-4bb5-b7a0-613d3aed474d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '528fc09c1cde4b50ab91560be809d9a6'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:20.909867 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1379/2776] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:42:20 2022] DELETE /identity/v3/users/528fc09c1cde4b50ab91560be809d9a6 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:20.914223 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da7d8f-4f0c-4825-9115-9fe12f00e9ab None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:20.914842 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da7d8f-4f0c-4825-9115-9fe12f00e9ab None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:20.914842 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da7d8f-4f0c-4825-9115-9fe12f00e9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:20.933548 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-79da7d8f-4f0c-4825-9115-9fe12f00e9ab None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:20.934932 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-79da7d8f-4f0c-4825-9115-9fe12f00e9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:20.987279 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87d455-64bc-440c-8190-47d31cf5eb04 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:20.987687 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87d455-64bc-440c-8190-47d31cf5eb04 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:20.989581 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87d455-64bc-440c-8190-47d31cf5eb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:21.001757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1398/2777] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:21.015289 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da7d8f-4f0c-4825-9115-9fe12f00e9ab None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:21.017974 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-79da7d8f-4f0c-4825-9115-9fe12f00e9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:21.025326 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6f87d455-64bc-440c-8190-47d31cf5eb04 None None] MFA Rules not processed for user `f58941916ebe4a68a9a9a2b04b745ee2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:21.026579 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6f87d455-64bc-440c-8190-47d31cf5eb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:21.085884 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da7d8f-4f0c-4825-9115-9fe12f00e9ab None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:21.090593 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202b1d0-644a-452f-a31a-6ef7dba66474 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:21.091192 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202b1d0-644a-452f-a31a-6ef7dba66474 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:21.093757 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202b1d0-644a-452f-a31a-6ef7dba66474 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:21.094069 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202b1d0-644a-452f-a31a-6ef7dba66474 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:21.104802 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202b1d0-644a-452f-a31a-6ef7dba66474 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:21.106285 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202b1d0-644a-452f-a31a-6ef7dba66474 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:21.110157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1380/2778] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:21.119740 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87d455-64bc-440c-8190-47d31cf5eb04 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:21.121969 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6f87d455-64bc-440c-8190-47d31cf5eb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:21.122510 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1399/2779] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:42:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:42:21.129763 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202b1d0-644a-452f-a31a-6ef7dba66474 service neutron] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:21.132018 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d202b1d0-644a-452f-a31a-6ef7dba66474 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:21.148762 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202b1d0-644a-452f-a31a-6ef7dba66474 service neutron] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:21.152407 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7771bf-7ec0-43b8-ab6f-a92d4a02c5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:21.153537 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7771bf-7ec0-43b8-ab6f-a92d4a02c5f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:21.153903 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7771bf-7ec0-43b8-ab6f-a92d4a02c5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:21.154287 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7771bf-7ec0-43b8-ab6f-a92d4a02c5f3 admin admin] PATH_INFO: `/v3/users/38fedc0d90b54e44978fc79b5a532ca0` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:21.156526 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7771bf-7ec0-43b8-ab6f-a92d4a02c5f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38fedc0d90b54e44978fc79b5a532ca0)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:21.158684 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7771bf-7ec0-43b8-ab6f-a92d4a02c5f3 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:21.193873 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87d455-64bc-440c-8190-47d31cf5eb04 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:21.197656 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02578fec-5288-404b-98d9-41c01e6af569 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:21.198318 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02578fec-5288-404b-98d9-41c01e6af569 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:21.198598 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02578fec-5288-404b-98d9-41c01e6af569 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:21.198876 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02578fec-5288-404b-98d9-41c01e6af569 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:21.208265 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02578fec-5288-404b-98d9-41c01e6af569 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:21.209742 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02578fec-5288-404b-98d9-41c01e6af569 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:21.219317 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1381/2780] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:42:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:42:21.400662 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ea7771bf-7ec0-43b8-ab6f-a92d4a02c5f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38fedc0d90b54e44978fc79b5a532ca0'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:21.401448 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ea7771bf-7ec0-43b8-ab6f-a92d4a02c5f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38fedc0d90b54e44978fc79b5a532ca0'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:21.413346 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ea7771bf-7ec0-43b8-ab6f-a92d4a02c5f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38fedc0d90b54e44978fc79b5a532ca0'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:21.415240 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:21.524123 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ea7771bf-7ec0-43b8-ab6f-a92d4a02c5f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38fedc0d90b54e44978fc79b5a532ca0'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:21.524747 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ea7771bf-7ec0-43b8-ab6f-a92d4a02c5f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38fedc0d90b54e44978fc79b5a532ca0'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:21.548391 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1400/2781] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:42:21 2022] DELETE /identity/v3/users/38fedc0d90b54e44978fc79b5a532ca0 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:42:21.551312 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b77044-f54d-4b7d-96fa-30487bff546c None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:21.553877 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b77044-f54d-4b7d-96fa-30487bff546c None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:21.553877 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b77044-f54d-4b7d-96fa-30487bff546c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:21.572133 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-a8b77044-f54d-4b7d-96fa-30487bff546c None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:21.573440 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a8b77044-f54d-4b7d-96fa-30487bff546c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:21.601202 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7771bf-7ec0-43b8-ab6f-a92d4a02c5f3 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:21.602944 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ea7771bf-7ec0-43b8-ab6f-a92d4a02c5f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:21.622085 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:21.658689 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1382/2782] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:42:21.669325 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b77044-f54d-4b7d-96fa-30487bff546c None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:21.671631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a8b77044-f54d-4b7d-96fa-30487bff546c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:21.686249 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7771bf-7ec0-43b8-ab6f-a92d4a02c5f3 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:21.691996 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21822e6e-a9ef-4d95-bffc-4cba4a42ae17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:21.693370 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21822e6e-a9ef-4d95-bffc-4cba4a42ae17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:21.693671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21822e6e-a9ef-4d95-bffc-4cba4a42ae17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:21.694038 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21822e6e-a9ef-4d95-bffc-4cba4a42ae17 admin admin] PATH_INFO: `/v3/projects/41a84a287aad4e089bf30d9c8fb91986` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:21.695450 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21822e6e-a9ef-4d95-bffc-4cba4a42ae17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41a84a287aad4e089bf30d9c8fb91986)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:21.697721 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21822e6e-a9ef-4d95-bffc-4cba4a42ae17 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:21.732647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b77044-f54d-4b7d-96fa-30487bff546c None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:21.737349 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9633e4d7-5c1a-445a-884c-328784d17cea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:21.738127 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633e4d7-5c1a-445a-884c-328784d17cea service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:21.738426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633e4d7-5c1a-445a-884c-328784d17cea service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:21.738705 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9633e4d7-5c1a-445a-884c-328784d17cea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:21.748823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633e4d7-5c1a-445a-884c-328784d17cea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:21.750400 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9633e4d7-5c1a-445a-884c-328784d17cea service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:21.760672 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1383/2783] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:42:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:42:21.998247 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-21822e6e-a9ef-4d95-bffc-4cba4a42ae17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41a84a287aad4e089bf30d9c8fb91986', 'request_id': 'req-21822e6e-a9ef-4d95-bffc-4cba4a42ae17'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:22.105436 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-21822e6e-a9ef-4d95-bffc-4cba4a42ae17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41a84a287aad4e089bf30d9c8fb91986', 'request_id': 'req-21822e6e-a9ef-4d95-bffc-4cba4a42ae17'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:22.113745 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-21822e6e-a9ef-4d95-bffc-4cba4a42ae17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41a84a287aad4e089bf30d9c8fb91986', 'request_id': 'req-21822e6e-a9ef-4d95-bffc-4cba4a42ae17'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:22.127431 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-21822e6e-a9ef-4d95-bffc-4cba4a42ae17 admin admin] The token cache is being invalidate because project 41a84a287aad4e089bf30d9c8fb91986 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:42:22.127765 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-21822e6e-a9ef-4d95-bffc-4cba4a42ae17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:22.131903 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-21822e6e-a9ef-4d95-bffc-4cba4a42ae17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:22.134254 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1401/2784] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:42:21 2022] DELETE /identity/v3/projects/41a84a287aad4e089bf30d9c8fb91986 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:42:22.139103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818538d-af91-40b6-9c71-88b8f49c137d None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:22.142189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818538d-af91-40b6-9c71-88b8f49c137d None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:22.142189 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818538d-af91-40b6-9c71-88b8f49c137d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:22.161121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c818538d-af91-40b6-9c71-88b8f49c137d None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:22.162378 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c818538d-af91-40b6-9c71-88b8f49c137d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:22.226143 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1384/2785] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:22.234514 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdc763-3b52-4b86-bc66-bd8177fb5c18 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:22.234897 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdc763-3b52-4b86-bc66-bd8177fb5c18 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:22.235250 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdc763-3b52-4b86-bc66-bd8177fb5c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:22.254782 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-bccdc763-3b52-4b86-bc66-bd8177fb5c18 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:22.256051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bccdc763-3b52-4b86-bc66-bd8177fb5c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:22.263865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1402/2786] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:22.268993 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c818538d-af91-40b6-9c71-88b8f49c137d None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:22.278647 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccdc763-3b52-4b86-bc66-bd8177fb5c18 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:22.280556 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-bccdc763-3b52-4b86-bc66-bd8177fb5c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:22.281611 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c818538d-af91-40b6-9c71-88b8f49c137d None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:22.288980 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b5d4e-d47b-4cd4-9fe9-02b203c4cb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:22.290198 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b5d4e-d47b-4cd4-9fe9-02b203c4cb60 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:22.290489 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b5d4e-d47b-4cd4-9fe9-02b203c4cb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:22.291078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b5d4e-d47b-4cd4-9fe9-02b203c4cb60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:22.291525 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b5d4e-d47b-4cd4-9fe9-02b203c4cb60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80297) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:42:22.291833 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b5d4e-d47b-4cd4-9fe9-02b203c4cb60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:22.293444 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b5d4e-d47b-4cd4-9fe9-02b203c4cb60 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:22.296532 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccdc763-3b52-4b86-bc66-bd8177fb5c18 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:22.310504 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1385/2787] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:42:22.310748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e76e8f-d13d-4b29-a0a5-0c18287c0a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:22.310748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e76e8f-d13d-4b29-a0a5-0c18287c0a60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:22.310748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e76e8f-d13d-4b29-a0a5-0c18287c0a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:22.310748 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e76e8f-d13d-4b29-a0a5-0c18287c0a60 admin admin] PATH_INFO: `/v3/projects/79ea8306f28d4c838e5b82645df11a21` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:22.311959 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e76e8f-d13d-4b29-a0a5-0c18287c0a60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79ea8306f28d4c838e5b82645df11a21)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:22.312541 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b5d4e-d47b-4cd4-9fe9-02b203c4cb60 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:22.314359 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e76e8f-d13d-4b29-a0a5-0c18287c0a60 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:22.323100 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b5d4e-d47b-4cd4-9fe9-02b203c4cb60 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:22.328701 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813290fa-4a90-41e2-a648-f4abf3a06300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:22.329363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813290fa-4a90-41e2-a648-f4abf3a06300 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:22.329667 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813290fa-4a90-41e2-a648-f4abf3a06300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:22.329940 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813290fa-4a90-41e2-a648-f4abf3a06300 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:22.330709 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813290fa-4a90-41e2-a648-f4abf3a06300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:22.332645 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813290fa-4a90-41e2-a648-f4abf3a06300 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:22.444327 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1386/2788] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:42:22 2022] POST /identity/v3/projects => generated 365 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:42:22.445366 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813290fa-4a90-41e2-a648-f4abf3a06300 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:22.452401 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-88e76e8f-d13d-4b29-a0a5-0c18287c0a60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79ea8306f28d4c838e5b82645df11a21', 'request_id': 'req-88e76e8f-d13d-4b29-a0a5-0c18287c0a60'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:22.456003 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:INSERT=3,keystone:DELETE=14,keystone:SELECT=83 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:22.506451 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813290fa-4a90-41e2-a648-f4abf3a06300 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:22.511137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd7dd7-9296-46b0-8ccc-0d382dd72ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:22.511852 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd7dd7-9296-46b0-8ccc-0d382dd72ab8 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:22.512140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd7dd7-9296-46b0-8ccc-0d382dd72ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:22.512417 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd7dd7-9296-46b0-8ccc-0d382dd72ab8 admin admin] PATH_INFO: `/v3/users` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:22.513103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd7dd7-9296-46b0-8ccc-0d382dd72ab8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:22.515382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd7dd7-9296-46b0-8ccc-0d382dd72ab8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:22.540977 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-88e76e8f-d13d-4b29-a0a5-0c18287c0a60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79ea8306f28d4c838e5b82645df11a21', 'request_id': 'req-88e76e8f-d13d-4b29-a0a5-0c18287c0a60'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:22.548819 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-88e76e8f-d13d-4b29-a0a5-0c18287c0a60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79ea8306f28d4c838e5b82645df11a21', 'request_id': 'req-88e76e8f-d13d-4b29-a0a5-0c18287c0a60'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:22.560407 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-88e76e8f-d13d-4b29-a0a5-0c18287c0a60 admin admin] The token cache is being invalidate because project 79ea8306f28d4c838e5b82645df11a21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:42:22.560709 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-88e76e8f-d13d-4b29-a0a5-0c18287c0a60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:22.561385 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-88e76e8f-d13d-4b29-a0a5-0c18287c0a60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:22.564394 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1403/2789] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:42:22 2022] DELETE /identity/v3/projects/79ea8306f28d4c838e5b82645df11a21 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:42:22.570075 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b49da7-e124-482c-843f-88be7d2f050e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:22.570365 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b49da7-e124-482c-843f-88be7d2f050e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:22.570644 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b49da7-e124-482c-843f-88be7d2f050e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:22.589590 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-83b49da7-e124-482c-843f-88be7d2f050e None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:22.590828 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-83b49da7-e124-482c-843f-88be7d2f050e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:22.599554 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1404/2790] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:42:22.608176 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5b2e6-b88c-4f05-b0e8-7c62cf3b553d None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:22.609025 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5b2e6-b88c-4f05-b0e8-7c62cf3b553d None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:22.609808 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5b2e6-b88c-4f05-b0e8-7c62cf3b553d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:22.624982 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1387/2791] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:42:22 2022] POST /identity/v3/users => generated 283 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:42:22.626260 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd7dd7-9296-46b0-8ccc-0d382dd72ab8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:22.628140 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c4bd7dd7-9296-46b0-8ccc-0d382dd72ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:22.628732 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-c2d5b2e6-b88c-4f05-b0e8-7c62cf3b553d None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:22.629902 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-c2d5b2e6-b88c-4f05-b0e8-7c62cf3b553d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:22.641960 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1405/2792] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:42:22.644843 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd7dd7-9296-46b0-8ccc-0d382dd72ab8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:22.646118 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5b2e6-b88c-4f05-b0e8-7c62cf3b553d None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:22.649792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d511e-e8de-4c55-941a-781730aae8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:22.650499 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d511e-e8de-4c55-941a-781730aae8b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:22.650780 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d511e-e8de-4c55-941a-781730aae8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:22.651058 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d511e-e8de-4c55-941a-781730aae8b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:22.651874 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d511e-e8de-4c55-941a-781730aae8b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:22.653504 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d511e-e8de-4c55-941a-781730aae8b2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:22.655520 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5b2e6-b88c-4f05-b0e8-7c62cf3b553d None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:22.660705 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea8284-8416-40d3-a222-f18237dca27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:22.661354 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea8284-8416-40d3-a222-f18237dca27d admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:22.661634 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea8284-8416-40d3-a222-f18237dca27d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:22.661910 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dea8284-8416-40d3-a222-f18237dca27d admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:22.662623 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea8284-8416-40d3-a222-f18237dca27d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:42:22.662927 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea8284-8416-40d3-a222-f18237dca27d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:22.667358 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1388/2793] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:42:22.667573 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dea8284-8416-40d3-a222-f18237dca27d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:22.668254 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d511e-e8de-4c55-941a-781730aae8b2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:22.678881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d511e-e8de-4c55-941a-781730aae8b2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:22.679873 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1406/2794] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 16:42:22.684409 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea8284-8416-40d3-a222-f18237dca27d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:22.686637 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88b9fb-717b-4031-92c2-2ce9a4bfc123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:22.688105 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88b9fb-717b-4031-92c2-2ce9a4bfc123 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:22.688575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88b9fb-717b-4031-92c2-2ce9a4bfc123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:22.688865 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88b9fb-717b-4031-92c2-2ce9a4bfc123 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:22.689594 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88b9fb-717b-4031-92c2-2ce9a4bfc123 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:22.691207 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88b9fb-717b-4031-92c2-2ce9a4bfc123 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:22.696108 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dea8284-8416-40d3-a222-f18237dca27d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:22.699394 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:22.701148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd651e-e5af-48f4-a932-c48439029c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:22.701809 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd651e-e5af-48f4-a932-c48439029c7d admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:22.702097 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd651e-e5af-48f4-a932-c48439029c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:22.702479 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd651e-e5af-48f4-a932-c48439029c7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:22.703171 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd651e-e5af-48f4-a932-c48439029c7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:22.705209 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1389/2795] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:42:22.711390 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd651e-e5af-48f4-a932-c48439029c7d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:22.714535 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88b9fb-717b-4031-92c2-2ce9a4bfc123 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:22.725178 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88b9fb-717b-4031-92c2-2ce9a4bfc123 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:22.729812 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7cf6e8-4dd9-478f-83aa-9b7c903386d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:22.730692 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7cf6e8-4dd9-478f-83aa-9b7c903386d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:22.730976 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7cf6e8-4dd9-478f-83aa-9b7c903386d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:22.731253 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7cf6e8-4dd9-478f-83aa-9b7c903386d0 admin admin] PATH_INFO: `/v3/projects/a4a685a329674c4dbe0bc673956f8827/users/8dfdd2eb990b40bfbcffffc4ea6c5da1/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:22.747104 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7cf6e8-4dd9-478f-83aa-9b7c903386d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4a685a329674c4dbe0bc673956f8827, user_id=8dfdd2eb990b40bfbcffffc4ea6c5da1, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:22.750717 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7cf6e8-4dd9-478f-83aa-9b7c903386d0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:22.810192 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1407/2796] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:42:22 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:42:22.814392 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd651e-e5af-48f4-a932-c48439029c7d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:22.830840 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1390/2797] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:42:22 2022] PUT /identity/v3/projects/a4a685a329674c4dbe0bc673956f8827/users/8dfdd2eb990b40bfbcffffc4ea6c5da1/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:42:22.845283 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497f65f-6955-4171-b726-090d1504791a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:22.845573 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497f65f-6955-4171-b726-090d1504791a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:22.845851 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497f65f-6955-4171-b726-090d1504791a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:22.875161 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8497f65f-6955-4171-b726-090d1504791a None None] MFA Rules not processed for user `8dfdd2eb990b40bfbcffffc4ea6c5da1`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:22.876428 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8497f65f-6955-4171-b726-090d1504791a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:22.881894 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd651e-e5af-48f4-a932-c48439029c7d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:22.938571 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5f018-f3cb-4208-859a-dcde33971959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:22.939249 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5f018-f3cb-4208-859a-dcde33971959 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:22.939583 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5f018-f3cb-4208-859a-dcde33971959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:22.939867 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5f018-f3cb-4208-859a-dcde33971959 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:22.940563 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac5f018-f3cb-4208-859a-dcde33971959 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:22.942766 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac5f018-f3cb-4208-859a-dcde33971959 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:22.944032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1391/2798] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:22 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:42:22.956371 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497f65f-6955-4171-b726-090d1504791a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:22.957903 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8497f65f-6955-4171-b726-090d1504791a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:23.016601 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497f65f-6955-4171-b726-090d1504791a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:23.020311 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb9c1b-29b1-4d14-8bf7-eb99c99e1ce5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:23.020962 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb9c1b-29b1-4d14-8bf7-eb99c99e1ce5 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:23.021246 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb9c1b-29b1-4d14-8bf7-eb99c99e1ce5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:23.021527 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb9c1b-29b1-4d14-8bf7-eb99c99e1ce5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:23.031192 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb9c1b-29b1-4d14-8bf7-eb99c99e1ce5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:23.032761 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb9c1b-29b1-4d14-8bf7-eb99c99e1ce5 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:23.037514 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1408/2799] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:42:22 2022] POST /identity/v3/users => generated 292 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:42:23.041555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5f018-f3cb-4208-859a-dcde33971959 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:23.042525 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1392/2800] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:42:22 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:42:23.058293 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5f018-f3cb-4208-859a-dcde33971959 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:23.065905 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e2633-5136-477a-befe-b0851ab59617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:23.066889 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e2633-5136-477a-befe-b0851ab59617 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:23.067485 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e2633-5136-477a-befe-b0851ab59617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:23.073844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e2633-5136-477a-befe-b0851ab59617 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:23.073844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e2633-5136-477a-befe-b0851ab59617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:23.073844 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e2633-5136-477a-befe-b0851ab59617 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:23.090005 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1409/2801] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 16:42:23.094619 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb9c1b-29b1-4d14-8bf7-eb99c99e1ce5 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:23.106255 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb9c1b-29b1-4d14-8bf7-eb99c99e1ce5 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:23.111743 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511f75f-55e4-4347-8259-23f92d20b743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:23.112413 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511f75f-55e4-4347-8259-23f92d20b743 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:23.112696 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511f75f-55e4-4347-8259-23f92d20b743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:23.112975 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511f75f-55e4-4347-8259-23f92d20b743 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:23.113734 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0511f75f-55e4-4347-8259-23f92d20b743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:23.115338 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0511f75f-55e4-4347-8259-23f92d20b743 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:23.126822 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1393/2802] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:42:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:42:23.130575 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e2633-5136-477a-befe-b0851ab59617 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:23.143595 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e2633-5136-477a-befe-b0851ab59617 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:23.149889 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73251b94-8c5a-4110-ac26-ebd5d60f15e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:23.154267 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73251b94-8c5a-4110-ac26-ebd5d60f15e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:23.154267 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73251b94-8c5a-4110-ac26-ebd5d60f15e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:23.154267 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73251b94-8c5a-4110-ac26-ebd5d60f15e8 admin admin] PATH_INFO: `/v3/projects/ce9b910eae974562929f660ba8c7af4c/users/9c4e7816dc9c4eeab652534160c22757/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:23.171423 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73251b94-8c5a-4110-ac26-ebd5d60f15e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce9b910eae974562929f660ba8c7af4c, user_id=9c4e7816dc9c4eeab652534160c22757, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:23.176112 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73251b94-8c5a-4110-ac26-ebd5d60f15e8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:23.198958 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9534e3b-2010-472c-9a06-6e12c6c8b842 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:23.199461 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9534e3b-2010-472c-9a06-6e12c6c8b842 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:23.199915 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9534e3b-2010-472c-9a06-6e12c6c8b842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:23.219847 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-b9534e3b-2010-472c-9a06-6e12c6c8b842 None None] MFA Rules not processed for user `f58941916ebe4a68a9a9a2b04b745ee2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:23.221060 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b9534e3b-2010-472c-9a06-6e12c6c8b842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:23.277551 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1394/2803] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:42:23.281841 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1410/2804] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:42:23 2022] PUT /identity/v3/projects/ce9b910eae974562929f660ba8c7af4c/users/9c4e7816dc9c4eeab652534160c22757/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:23.286834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3d7f7-447b-4713-be90-6667c3021d05 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:23.287134 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3d7f7-447b-4713-be90-6667c3021d05 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:23.287493 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3d7f7-447b-4713-be90-6667c3021d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:23.305864 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-94d3d7f7-447b-4713-be90-6667c3021d05 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:23.307958 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-94d3d7f7-447b-4713-be90-6667c3021d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:23.372789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1395/2805] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:23.382450 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73251b94-8c5a-4110-ac26-ebd5d60f15e8 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:23.384343 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-73251b94-8c5a-4110-ac26-ebd5d60f15e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:23.399167 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:23.444501 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73251b94-8c5a-4110-ac26-ebd5d60f15e8 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:23.456087 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b9859-f1e7-419d-b8a8-68882e29cb61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:23.456087 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b9859-f1e7-419d-b8a8-68882e29cb61 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:23.456087 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b9859-f1e7-419d-b8a8-68882e29cb61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:23.456087 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b9859-f1e7-419d-b8a8-68882e29cb61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:23.460770 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b9859-f1e7-419d-b8a8-68882e29cb61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:23.462277 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b9859-f1e7-419d-b8a8-68882e29cb61 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:23.473854 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1411/2806] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:42:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:42:23.489153 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:29.128644 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3d7f7-447b-4713-be90-6667c3021d05 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:29.138754 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-94d3d7f7-447b-4713-be90-6667c3021d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:29.159903 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3d7f7-447b-4713-be90-6667c3021d05 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:29.170918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b86168-87a2-4d7d-86bf-db286cada743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:29.171743 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b86168-87a2-4d7d-86bf-db286cada743 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:29.172078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b86168-87a2-4d7d-86bf-db286cada743 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:29.172458 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b86168-87a2-4d7d-86bf-db286cada743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:29.175382 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-a8b86168-87a2-4d7d-86bf-db286cada743 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:29.277320 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b86168-87a2-4d7d-86bf-db286cada743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:29.278446 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b86168-87a2-4d7d-86bf-db286cada743 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:29.291954 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1396/2807] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:42:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:42:30.948080 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04141bb5-35d8-401a-bed7-65bf40b514a2 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:30.949114 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04141bb5-35d8-401a-bed7-65bf40b514a2 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:30.949578 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04141bb5-35d8-401a-bed7-65bf40b514a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:30.981643 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-04141bb5-35d8-401a-bed7-65bf40b514a2 None None] MFA Rules not processed for user `9c4e7816dc9c4eeab652534160c22757`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:30.982422 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-04141bb5-35d8-401a-bed7-65bf40b514a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:31.058375 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1412/2808] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:31.063845 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b86168-87a2-4d7d-86bf-db286cada743 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:31.076766 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b86168-87a2-4d7d-86bf-db286cada743 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:31.084638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db888f-9030-4ac2-aa60-728750d56c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:31.085756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db888f-9030-4ac2-aa60-728750d56c93 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:31.086229 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db888f-9030-4ac2-aa60-728750d56c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:31.087388 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db888f-9030-4ac2-aa60-728750d56c93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:31.088067 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db888f-9030-4ac2-aa60-728750d56c93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:31.090170 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99db888f-9030-4ac2-aa60-728750d56c93 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:31.277951 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1397/2809] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:42:31 2022] POST /identity/v3/projects => generated 385 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:42:31.279126 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04141bb5-35d8-401a-bed7-65bf40b514a2 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:31.296199 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04141bb5-35d8-401a-bed7-65bf40b514a2 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:31.411534 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d651e-07bb-40cc-a22c-ba71eacefb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:31.413109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d651e-07bb-40cc-a22c-ba71eacefb51 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:31.413831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d651e-07bb-40cc-a22c-ba71eacefb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:31.414370 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d651e-07bb-40cc-a22c-ba71eacefb51 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:31.415542 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1d651e-07bb-40cc-a22c-ba71eacefb51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:31.419716 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1d651e-07bb-40cc-a22c-ba71eacefb51 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:31.516406 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1413/2810] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:42:31 2022] POST /identity/v3/users => generated 299 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:42:31.517835 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db888f-9030-4ac2-aa60-728750d56c93 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:31.536792 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db888f-9030-4ac2-aa60-728750d56c93 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:31.555406 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae1069-4300-4230-ad36-95b506143fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:31.556285 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae1069-4300-4230-ad36-95b506143fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:31.556944 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae1069-4300-4230-ad36-95b506143fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:31.556944 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ae1069-4300-4230-ad36-95b506143fb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:31.557808 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ae1069-4300-4230-ad36-95b506143fb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:31.559484 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ae1069-4300-4230-ad36-95b506143fb3 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:31.581574 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1398/2811] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:42:31.585372 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d651e-07bb-40cc-a22c-ba71eacefb51 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:31.603922 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d651e-07bb-40cc-a22c-ba71eacefb51 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:31.617835 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf2793-a3db-42c4-9594-f5b2e3a4cee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:31.618756 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf2793-a3db-42c4-9594-f5b2e3a4cee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:31.619390 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf2793-a3db-42c4-9594-f5b2e3a4cee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:31.620327 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbf2793-a3db-42c4-9594-f5b2e3a4cee4 admin admin] PATH_INFO: `/v3/projects/00671e957c8644e6954a41aa309e8608/users/631ba6e9064243a58465648949c50820/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:31.646549 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:31.651326 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf2793-a3db-42c4-9594-f5b2e3a4cee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00671e957c8644e6954a41aa309e8608, user_id=631ba6e9064243a58465648949c50820, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:31.654440 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbf2793-a3db-42c4-9594-f5b2e3a4cee4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:31.787997 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1414/2812] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:42:31 2022] PUT /identity/v3/projects/00671e957c8644e6954a41aa309e8608/users/631ba6e9064243a58465648949c50820/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:31.788161 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae1069-4300-4230-ad36-95b506143fb3 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:31.804425 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ae1069-4300-4230-ad36-95b506143fb3 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:31.821890 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:31.900570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac522c3-8fe0-446f-836c-a4ad7fea98c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:31.901610 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac522c3-8fe0-446f-836c-a4ad7fea98c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:31.902167 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac522c3-8fe0-446f-836c-a4ad7fea98c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:31.902515 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac522c3-8fe0-446f-836c-a4ad7fea98c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:31.904978 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac522c3-8fe0-446f-836c-a4ad7fea98c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:31.908065 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac522c3-8fe0-446f-836c-a4ad7fea98c2 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:31.922834 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1399/2813] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:42:31.926986 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf2793-a3db-42c4-9594-f5b2e3a4cee4 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:31.943159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbf2793-a3db-42c4-9594-f5b2e3a4cee4 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:31.948308 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8404b-b40e-4160-907c-5dd33644a7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:31.949303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8404b-b40e-4160-907c-5dd33644a7fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:31.949665 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8404b-b40e-4160-907c-5dd33644a7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:31.950018 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8404b-b40e-4160-907c-5dd33644a7fd admin admin] PATH_INFO: `/v3/projects/00671e957c8644e6954a41aa309e8608/users/631ba6e9064243a58465648949c50820/roles/8acc34a163874a60b03728b814eaf075` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:31.952204 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb8404b-b40e-4160-907c-5dd33644a7fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=00671e957c8644e6954a41aa309e8608, user_id=631ba6e9064243a58465648949c50820, role_id=8acc34a163874a60b03728b814eaf075)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:31.955018 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb8404b-b40e-4160-907c-5dd33644a7fd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:31.966218 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1415/2814] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:42:31 2022] PUT /identity/v3/projects/00671e957c8644e6954a41aa309e8608/users/631ba6e9064243a58465648949c50820/roles/8acc34a163874a60b03728b814eaf075 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:39.036922 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac522c3-8fe0-446f-836c-a4ad7fea98c2 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:39.167292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd63f8-7101-4274-90cb-ee9a2daf95c0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:39.167807 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd63f8-7101-4274-90cb-ee9a2daf95c0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:39.168009 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fd63f8-7101-4274-90cb-ee9a2daf95c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:39.183035 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac522c3-8fe0-446f-836c-a4ad7fea98c2 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:39.229636 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-82fd63f8-7101-4274-90cb-ee9a2daf95c0 None None] MFA Rules not processed for user `631ba6e9064243a58465648949c50820`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:39.229636 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-82fd63f8-7101-4274-90cb-ee9a2daf95c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:39.254705 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9b2832-6683-4743-8468-e81d694a996f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:39.255539 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b2832-6683-4743-8468-e81d694a996f service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:39.255829 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b2832-6683-4743-8468-e81d694a996f service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:39.256112 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b2832-6683-4743-8468-e81d694a996f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:39.258222 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5f9b2832-6683-4743-8468-e81d694a996f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:39.345280 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b2832-6683-4743-8468-e81d694a996f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:39.346738 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9b2832-6683-4743-8468-e81d694a996f service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:39.347129 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1416/2815] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:39.369121 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fd63f8-7101-4274-90cb-ee9a2daf95c0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:39.370501 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1400/2816] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:42:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:42:39.371584 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-82fd63f8-7101-4274-90cb-ee9a2daf95c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:39.462596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fd63f8-7101-4274-90cb-ee9a2daf95c0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:39.468097 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac005c-5e99-4fb3-9e7e-2fed58514f6f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec41c84c9375437590c42621ca5b400a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:39.468783 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac005c-5e99-4fb3-9e7e-2fed58514f6f service glance] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:39.469660 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac005c-5e99-4fb3-9e7e-2fed58514f6f service glance] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:39.469913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac005c-5e99-4fb3-9e7e-2fed58514f6f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:39.538800 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac005c-5e99-4fb3-9e7e-2fed58514f6f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:39.540566 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac005c-5e99-4fb3-9e7e-2fed58514f6f service glance] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:39.571769 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1417/2817] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:42:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:42:39.866925 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06352e46-f503-45a9-97b9-b0608dbe16a0 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:39.867216 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06352e46-f503-45a9-97b9-b0608dbe16a0 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:39.867529 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06352e46-f503-45a9-97b9-b0608dbe16a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:39.890044 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-06352e46-f503-45a9-97b9-b0608dbe16a0 None None] MFA Rules not processed for user `d49604bffa3d4e89a631eefa6d4e4f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:39.891336 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-06352e46-f503-45a9-97b9-b0608dbe16a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:39.921801 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:39.969591 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1401/2818] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 30 16:42:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:40.113504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac005c-5e99-4fb3-9e7e-2fed58514f6f service glance] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:40.115081 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-93ac005c-5e99-4fb3-9e7e-2fed58514f6f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:40.137176 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:40.191305 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac005c-5e99-4fb3-9e7e-2fed58514f6f service glance] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:40.197011 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90803beb-1fd7-4af2-bb90-970e15a646f7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988b917eea4b48ffae32e23b8555a5c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:40.197672 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90803beb-1fd7-4af2-bb90-970e15a646f7 service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:40.197951 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90803beb-1fd7-4af2-bb90-970e15a646f7 service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:40.198229 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90803beb-1fd7-4af2-bb90-970e15a646f7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:40.208200 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90803beb-1fd7-4af2-bb90-970e15a646f7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:40.209860 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90803beb-1fd7-4af2-bb90-970e15a646f7 service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:40.223658 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1418/2819] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 30 16:42:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 16:42:45.356160 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6dd8e-8c67-4398-b3b9-aed49cd0da2a None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:45.356923 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6dd8e-8c67-4398-b3b9-aed49cd0da2a None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:45.356923 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6dd8e-8c67-4398-b3b9-aed49cd0da2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:45.382666 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f7b6dd8e-8c67-4398-b3b9-aed49cd0da2a None None] MFA Rules not processed for user `f58941916ebe4a68a9a9a2b04b745ee2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:45.384737 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f7b6dd8e-8c67-4398-b3b9-aed49cd0da2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:45.475133 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1402/2820] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:50.276333 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90803beb-1fd7-4af2-bb90-970e15a646f7 service swift] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:50.279512 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-90803beb-1fd7-4af2-bb90-970e15a646f7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:50.385325 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90803beb-1fd7-4af2-bb90-970e15a646f7 service swift] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:50.391917 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b2dba-d927-4ebe-967e-00b108556b97 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a932a9390fdd47f3adcf053a60250944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:50.392601 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b2dba-d927-4ebe-967e-00b108556b97 service placement] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:50.392887 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b2dba-d927-4ebe-967e-00b108556b97 service placement] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:50.393170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b2dba-d927-4ebe-967e-00b108556b97 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:50.395243 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2b7b2dba-d927-4ebe-967e-00b108556b97 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:50.410676 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b2dba-d927-4ebe-967e-00b108556b97 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:50.415499 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b2dba-d927-4ebe-967e-00b108556b97 service placement] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:50.432102 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b6dd8e-8c67-4398-b3b9-aed49cd0da2a None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:50.435105 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f7b6dd8e-8c67-4398-b3b9-aed49cd0da2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:50.443632 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1419/2821] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:42:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:42:50.464772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b6dd8e-8c67-4398-b3b9-aed49cd0da2a None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:50.475570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d607b4-ace6-4f63-8e9e-7e3270ea2990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:50.476429 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d607b4-ace6-4f63-8e9e-7e3270ea2990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:50.476736 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d607b4-ace6-4f63-8e9e-7e3270ea2990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:50.477022 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d607b4-ace6-4f63-8e9e-7e3270ea2990 admin admin] PATH_INFO: `/v3/users/b1a5d200007a4474a2a25d2ae0915c14` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:50.484935 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d607b4-ace6-4f63-8e9e-7e3270ea2990 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1a5d200007a4474a2a25d2ae0915c14)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:50.487100 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d607b4-ace6-4f63-8e9e-7e3270ea2990 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:50.756858 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e1d607b4-ace6-4f63-8e9e-7e3270ea2990 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1a5d200007a4474a2a25d2ae0915c14'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:50.757475 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e1d607b4-ace6-4f63-8e9e-7e3270ea2990 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1a5d200007a4474a2a25d2ae0915c14'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:50.850399 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e1d607b4-ace6-4f63-8e9e-7e3270ea2990 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1a5d200007a4474a2a25d2ae0915c14'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:50.865678 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e1d607b4-ace6-4f63-8e9e-7e3270ea2990 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1a5d200007a4474a2a25d2ae0915c14'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:50.866529 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e1d607b4-ace6-4f63-8e9e-7e3270ea2990 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1a5d200007a4474a2a25d2ae0915c14'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:50.902701 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1403/2822] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:42:50 2022] DELETE /identity/v3/users/b1a5d200007a4474a2a25d2ae0915c14 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:51.489711 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b2dba-d927-4ebe-967e-00b108556b97 service placement] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:51.511510 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b2dba-d927-4ebe-967e-00b108556b97 service placement] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:51.597838 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2408f1-6f19-4925-b5a1-f9bf3e75c6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:51.598737 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2408f1-6f19-4925-b5a1-f9bf3e75c6fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:51.599014 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2408f1-6f19-4925-b5a1-f9bf3e75c6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:51.599282 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2408f1-6f19-4925-b5a1-f9bf3e75c6fd admin admin] PATH_INFO: `/v3/projects/19b35fc495a345898167db62b9b95887` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:51.600510 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2408f1-6f19-4925-b5a1-f9bf3e75c6fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=19b35fc495a345898167db62b9b95887)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:51.602399 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2408f1-6f19-4925-b5a1-f9bf3e75c6fd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:51.717469 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:51.726669 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ba2408f1-6f19-4925-b5a1-f9bf3e75c6fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19b35fc495a345898167db62b9b95887', 'request_id': 'req-ba2408f1-6f19-4925-b5a1-f9bf3e75c6fd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:51.850129 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ba2408f1-6f19-4925-b5a1-f9bf3e75c6fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19b35fc495a345898167db62b9b95887', 'request_id': 'req-ba2408f1-6f19-4925-b5a1-f9bf3e75c6fd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:51.863954 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ba2408f1-6f19-4925-b5a1-f9bf3e75c6fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19b35fc495a345898167db62b9b95887', 'request_id': 'req-ba2408f1-6f19-4925-b5a1-f9bf3e75c6fd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:51.877469 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ba2408f1-6f19-4925-b5a1-f9bf3e75c6fd admin admin] The token cache is being invalidate because project 19b35fc495a345898167db62b9b95887 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:42:51.877698 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ba2408f1-6f19-4925-b5a1-f9bf3e75c6fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:51.878420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ba2408f1-6f19-4925-b5a1-f9bf3e75c6fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:51.883282 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1420/2823] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:42:51 2022] DELETE /identity/v3/projects/19b35fc495a345898167db62b9b95887 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:51.884459 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d607b4-ace6-4f63-8e9e-7e3270ea2990 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:51.887194 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-e1d607b4-ace6-4f63-8e9e-7e3270ea2990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:51.904290 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d607b4-ace6-4f63-8e9e-7e3270ea2990 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:51.952410 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:51.972488 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feb5859-663a-4813-9cd7-f52520444ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:51.973595 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb5859-663a-4813-9cd7-f52520444ccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:51.973888 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb5859-663a-4813-9cd7-f52520444ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:51.974308 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feb5859-663a-4813-9cd7-f52520444ccb admin admin] PATH_INFO: `/v3/users/23f851892cfa41d08c39d4929267946f` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:51.975752 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb5859-663a-4813-9cd7-f52520444ccb admin admin] RBAC: Authorizing `identity:delete_user(user_id=23f851892cfa41d08c39d4929267946f)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:51.978103 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feb5859-663a-4813-9cd7-f52520444ccb admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:52.517824 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7feb5859-663a-4813-9cd7-f52520444ccb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23f851892cfa41d08c39d4929267946f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:52.518414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7feb5859-663a-4813-9cd7-f52520444ccb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23f851892cfa41d08c39d4929267946f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:52.862057 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7feb5859-663a-4813-9cd7-f52520444ccb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23f851892cfa41d08c39d4929267946f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:52.870186 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7feb5859-663a-4813-9cd7-f52520444ccb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23f851892cfa41d08c39d4929267946f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:52.870854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-7feb5859-663a-4813-9cd7-f52520444ccb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23f851892cfa41d08c39d4929267946f'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:52.894600 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1404/2824] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:42:51 2022] DELETE /identity/v3/users/23f851892cfa41d08c39d4929267946f => generated 0 bytes in 1008 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:42:53.215563 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2408f1-6f19-4925-b5a1-f9bf3e75c6fd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:53.217804 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ba2408f1-6f19-4925-b5a1-f9bf3e75c6fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:53.235882 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2408f1-6f19-4925-b5a1-f9bf3e75c6fd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:53.325534 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3dc7ca-41f6-4be3-9f8e-2f77c9b3f6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:53.326755 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3dc7ca-41f6-4be3-9f8e-2f77c9b3f6d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:53.327247 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3dc7ca-41f6-4be3-9f8e-2f77c9b3f6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:53.327978 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3dc7ca-41f6-4be3-9f8e-2f77c9b3f6d8 admin admin] PATH_INFO: `/v3/projects/3c9e3028b2404fc09a0a270fab412a09` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:53.330128 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3dc7ca-41f6-4be3-9f8e-2f77c9b3f6d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c9e3028b2404fc09a0a270fab412a09)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:53.332811 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3dc7ca-41f6-4be3-9f8e-2f77c9b3f6d8 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:53.567196 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6f3dc7ca-41f6-4be3-9f8e-2f77c9b3f6d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c9e3028b2404fc09a0a270fab412a09', 'request_id': 'req-6f3dc7ca-41f6-4be3-9f8e-2f77c9b3f6d8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:53.716509 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6f3dc7ca-41f6-4be3-9f8e-2f77c9b3f6d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c9e3028b2404fc09a0a270fab412a09', 'request_id': 'req-6f3dc7ca-41f6-4be3-9f8e-2f77c9b3f6d8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:53.732534 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6f3dc7ca-41f6-4be3-9f8e-2f77c9b3f6d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c9e3028b2404fc09a0a270fab412a09', 'request_id': 'req-6f3dc7ca-41f6-4be3-9f8e-2f77c9b3f6d8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:53.742553 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6f3dc7ca-41f6-4be3-9f8e-2f77c9b3f6d8 admin admin] The token cache is being invalidate because project 3c9e3028b2404fc09a0a270fab412a09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:42:53.742849 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6f3dc7ca-41f6-4be3-9f8e-2f77c9b3f6d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:53.743572 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-6f3dc7ca-41f6-4be3-9f8e-2f77c9b3f6d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:42:53.745322 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1421/2825] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:42:53 2022] DELETE /identity/v3/projects/3c9e3028b2404fc09a0a270fab412a09 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:42:54.275954 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ca683-3154-4c2f-9cda-c8b3e23cf1d8 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:54.276578 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ca683-3154-4c2f-9cda-c8b3e23cf1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:54.276697 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ca683-3154-4c2f-9cda-c8b3e23cf1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:54.296535 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c04ca683-3154-4c2f-9cda-c8b3e23cf1d8 None None] MFA Rules not processed for user `9c4e7816dc9c4eeab652534160c22757`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:54.297881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c04ca683-3154-4c2f-9cda-c8b3e23cf1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:54.357650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1405/2826] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:42:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:42:59.090920 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ddd853-934f-4d5a-9229-57ef38d4f14f None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:59.091818 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ddd853-934f-4d5a-9229-57ef38d4f14f None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:59.092113 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ddd853-934f-4d5a-9229-57ef38d4f14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:59.116069 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-85ddd853-934f-4d5a-9229-57ef38d4f14f None None] MFA Rules not processed for user `d49604bffa3d4e89a631eefa6d4e4f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:42:59.117359 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-85ddd853-934f-4d5a-9229-57ef38d4f14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:59.180592 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1422/2827] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 30 16:42:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:42:59.198823 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ca683-3154-4c2f-9cda-c8b3e23cf1d8 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:42:59.200272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c04ca683-3154-4c2f-9cda-c8b3e23cf1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:42:59.277808 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ca683-3154-4c2f-9cda-c8b3e23cf1d8 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:42:59.283854 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5809f056-2a82-42be-be32-db1141b8c2f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988b917eea4b48ffae32e23b8555a5c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:42:59.284526 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809f056-2a82-42be-be32-db1141b8c2f9 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:42:59.284811 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809f056-2a82-42be-be32-db1141b8c2f9 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:42:59.285089 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5809f056-2a82-42be-be32-db1141b8c2f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:42:59.291883 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:42:59.294602 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5809f056-2a82-42be-be32-db1141b8c2f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:42:59.296252 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5809f056-2a82-42be-be32-db1141b8c2f9 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:42:59.307044 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1406/2828] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 30 16:42:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 16:43:05.201904 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ddd853-934f-4d5a-9229-57ef38d4f14f None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:05.203642 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-85ddd853-934f-4d5a-9229-57ef38d4f14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:05.208945 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:43:05.289132 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ddd853-934f-4d5a-9229-57ef38d4f14f None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:05.294992 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9052c0c-8b04-424c-a3fc-fbd14064afa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:05.296217 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9052c0c-8b04-424c-a3fc-fbd14064afa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:05.296608 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9052c0c-8b04-424c-a3fc-fbd14064afa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:05.296989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9052c0c-8b04-424c-a3fc-fbd14064afa4 admin admin] PATH_INFO: `/v3/users/8dfdd2eb990b40bfbcffffc4ea6c5da1` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:05.298930 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9052c0c-8b04-424c-a3fc-fbd14064afa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dfdd2eb990b40bfbcffffc4ea6c5da1)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:05.301143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9052c0c-8b04-424c-a3fc-fbd14064afa4 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:05.539170 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b9052c0c-8b04-424c-a3fc-fbd14064afa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dfdd2eb990b40bfbcffffc4ea6c5da1'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:05.539904 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b9052c0c-8b04-424c-a3fc-fbd14064afa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dfdd2eb990b40bfbcffffc4ea6c5da1'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:05.547420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b9052c0c-8b04-424c-a3fc-fbd14064afa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dfdd2eb990b40bfbcffffc4ea6c5da1'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:05.619497 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b9052c0c-8b04-424c-a3fc-fbd14064afa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dfdd2eb990b40bfbcffffc4ea6c5da1'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:05.620234 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-b9052c0c-8b04-424c-a3fc-fbd14064afa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dfdd2eb990b40bfbcffffc4ea6c5da1'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:05.639897 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1423/2829] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:43:05 2022] DELETE /identity/v3/users/8dfdd2eb990b40bfbcffffc4ea6c5da1 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:43:05.644748 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e92b5f-db89-4b3b-a4cf-c00716be6386 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:05.645244 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e92b5f-db89-4b3b-a4cf-c00716be6386 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:05.645363 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e92b5f-db89-4b3b-a4cf-c00716be6386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:05.664628 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-c2e92b5f-db89-4b3b-a4cf-c00716be6386 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:43:05.666211 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c2e92b5f-db89-4b3b-a4cf-c00716be6386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:05.746275 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1407/2830] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:43:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:43:05.789504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9052c0c-8b04-424c-a3fc-fbd14064afa4 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:05.789504 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-b9052c0c-8b04-424c-a3fc-fbd14064afa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:05.835904 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9052c0c-8b04-424c-a3fc-fbd14064afa4 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:05.854400 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d90ce0-c5e7-47bd-a0b8-7af9a9487a08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:05.855594 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d90ce0-c5e7-47bd-a0b8-7af9a9487a08 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:05.856131 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d90ce0-c5e7-47bd-a0b8-7af9a9487a08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:05.856667 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d90ce0-c5e7-47bd-a0b8-7af9a9487a08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:05.881213 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d90ce0-c5e7-47bd-a0b8-7af9a9487a08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:05.883021 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d90ce0-c5e7-47bd-a0b8-7af9a9487a08 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:05.904851 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1424/2831] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:43:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:43:07.083844 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e92b5f-db89-4b3b-a4cf-c00716be6386 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:07.085426 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-c2e92b5f-db89-4b3b-a4cf-c00716be6386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:07.120196 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e92b5f-db89-4b3b-a4cf-c00716be6386 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:07.128397 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448decbf-a78e-484d-9981-72d8f8c78513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:07.130024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448decbf-a78e-484d-9981-72d8f8c78513 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:07.130475 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448decbf-a78e-484d-9981-72d8f8c78513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:07.131028 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448decbf-a78e-484d-9981-72d8f8c78513 admin admin] PATH_INFO: `/v3/projects/a4a685a329674c4dbe0bc673956f8827` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:07.132507 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448decbf-a78e-484d-9981-72d8f8c78513 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4a685a329674c4dbe0bc673956f8827)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:07.134844 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448decbf-a78e-484d-9981-72d8f8c78513 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:07.308850 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-448decbf-a78e-484d-9981-72d8f8c78513 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4a685a329674c4dbe0bc673956f8827', 'request_id': 'req-448decbf-a78e-484d-9981-72d8f8c78513'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:07.324935 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-448decbf-a78e-484d-9981-72d8f8c78513 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4a685a329674c4dbe0bc673956f8827', 'request_id': 'req-448decbf-a78e-484d-9981-72d8f8c78513'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:07.334999 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-448decbf-a78e-484d-9981-72d8f8c78513 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4a685a329674c4dbe0bc673956f8827', 'request_id': 'req-448decbf-a78e-484d-9981-72d8f8c78513'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:07.492414 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-448decbf-a78e-484d-9981-72d8f8c78513 admin admin] The token cache is being invalidate because project a4a685a329674c4dbe0bc673956f8827 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:43:07.492706 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-448decbf-a78e-484d-9981-72d8f8c78513 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:07.493364 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-448decbf-a78e-484d-9981-72d8f8c78513 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:07.498369 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1408/2832] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:43:07 2022] DELETE /identity/v3/projects/a4a685a329674c4dbe0bc673956f8827 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:43:07.500133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf742c0-3bd3-4e09-a40b-14c6507f0e5e None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:07.500718 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf742c0-3bd3-4e09-a40b-14c6507f0e5e None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:07.500718 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf742c0-3bd3-4e09-a40b-14c6507f0e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:07.522864 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-7bf742c0-3bd3-4e09-a40b-14c6507f0e5e None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:43:07.524299 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-7bf742c0-3bd3-4e09-a40b-14c6507f0e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:07.544768 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:43:07.603514 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1425/2833] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:43:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:43:07.609875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941c524-5b2e-42da-a3f3-ad5d2c4320ed None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:07.613458 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941c524-5b2e-42da-a3f3-ad5d2c4320ed None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:07.613458 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941c524-5b2e-42da-a3f3-ad5d2c4320ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:07.633109 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-8941c524-5b2e-42da-a3f3-ad5d2c4320ed None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:43:07.634063 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8941c524-5b2e-42da-a3f3-ad5d2c4320ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:07.645640 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1409/2834] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:43:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:43:07.650456 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf742c0-3bd3-4e09-a40b-14c6507f0e5e None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:07.668982 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf742c0-3bd3-4e09-a40b-14c6507f0e5e None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:07.675546 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e4eaf2-0d2a-41ef-9dc1-4cad34e5940a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:07.676396 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4eaf2-0d2a-41ef-9dc1-4cad34e5940a admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:07.677160 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4eaf2-0d2a-41ef-9dc1-4cad34e5940a admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:07.677585 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4eaf2-0d2a-41ef-9dc1-4cad34e5940a admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:07.678591 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e4eaf2-0d2a-41ef-9dc1-4cad34e5940a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:43:07.679030 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e4eaf2-0d2a-41ef-9dc1-4cad34e5940a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:07.680966 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e4eaf2-0d2a-41ef-9dc1-4cad34e5940a admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:07.700463 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1426/2835] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:43:07.704586 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941c524-5b2e-42da-a3f3-ad5d2c4320ed None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:07.716290 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941c524-5b2e-42da-a3f3-ad5d2c4320ed None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:07.721585 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c6506-3969-4a4f-b837-399fd8c86633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:07.722269 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c6506-3969-4a4f-b837-399fd8c86633 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:07.722545 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c6506-3969-4a4f-b837-399fd8c86633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:07.722827 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c6506-3969-4a4f-b837-399fd8c86633 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:07.723938 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c6506-3969-4a4f-b837-399fd8c86633 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:07.725471 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c6506-3969-4a4f-b837-399fd8c86633 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:07.834623 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1410/2836] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:43:07 2022] POST /identity/v3/projects => generated 387 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:43:07.839423 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e4eaf2-0d2a-41ef-9dc1-4cad34e5940a admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:07.850555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e4eaf2-0d2a-41ef-9dc1-4cad34e5940a admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:07.944548 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60d4ad-7277-4620-9ad3-f924ff814b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:07.945208 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60d4ad-7277-4620-9ad3-f924ff814b54 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:07.945548 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60d4ad-7277-4620-9ad3-f924ff814b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:07.945838 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60d4ad-7277-4620-9ad3-f924ff814b54 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:07.946596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60d4ad-7277-4620-9ad3-f924ff814b54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:07.948591 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60d4ad-7277-4620-9ad3-f924ff814b54 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:08.037039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1427/2837] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:43:07 2022] POST /identity/v3/users => generated 294 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:43:08.040902 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c6506-3969-4a4f-b837-399fd8c86633 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:08.053051 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c6506-3969-4a4f-b837-399fd8c86633 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:08.060367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94513bc2-94e9-4d71-806e-e0ba6f36c11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:08.061131 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94513bc2-94e9-4d71-806e-e0ba6f36c11c admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:08.061498 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94513bc2-94e9-4d71-806e-e0ba6f36c11c admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:08.062004 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94513bc2-94e9-4d71-806e-e0ba6f36c11c admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:08.062735 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94513bc2-94e9-4d71-806e-e0ba6f36c11c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:08.065910 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94513bc2-94e9-4d71-806e-e0ba6f36c11c admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:08.078303 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1411/2838] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:43:08.083302 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60d4ad-7277-4620-9ad3-f924ff814b54 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:08.094588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60d4ad-7277-4620-9ad3-f924ff814b54 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:08.100607 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fed7eb-c80f-4ae9-b338-5210884681db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:08.101276 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fed7eb-c80f-4ae9-b338-5210884681db admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:08.101557 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fed7eb-c80f-4ae9-b338-5210884681db admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:08.101834 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fed7eb-c80f-4ae9-b338-5210884681db admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:08.102562 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fed7eb-c80f-4ae9-b338-5210884681db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:08.104289 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fed7eb-c80f-4ae9-b338-5210884681db admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:08.116688 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1428/2839] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:43:08.120723 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94513bc2-94e9-4d71-806e-e0ba6f36c11c admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:08.133986 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94513bc2-94e9-4d71-806e-e0ba6f36c11c admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:08.139241 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181a72a-b75a-461b-9e33-8c6379ad2e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:08.140186 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181a72a-b75a-461b-9e33-8c6379ad2e33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:08.140471 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181a72a-b75a-461b-9e33-8c6379ad2e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:08.140747 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181a72a-b75a-461b-9e33-8c6379ad2e33 admin admin] PATH_INFO: `/v3/projects/813b58b12d3f4b7a855d9c6c20e0664e/users/bad13e4e5a134b4fb2611c7aee3b0ba8/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:08.156762 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181a72a-b75a-461b-9e33-8c6379ad2e33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=813b58b12d3f4b7a855d9c6c20e0664e, user_id=bad13e4e5a134b4fb2611c7aee3b0ba8, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:08.159556 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181a72a-b75a-461b-9e33-8c6379ad2e33 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:08.256954 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1412/2840] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:43:08 2022] PUT /identity/v3/projects/813b58b12d3f4b7a855d9c6c20e0664e/users/bad13e4e5a134b4fb2611c7aee3b0ba8/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:43:08.276383 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9edcf6-1f80-4be9-b2bb-771ec69ae0c8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:08.280133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9edcf6-1f80-4be9-b2bb-771ec69ae0c8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:08.280133 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9edcf6-1f80-4be9-b2bb-771ec69ae0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:08.308475 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-5c9edcf6-1f80-4be9-b2bb-771ec69ae0c8 None None] MFA Rules not processed for user `bad13e4e5a134b4fb2611c7aee3b0ba8`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:43:08.310776 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5c9edcf6-1f80-4be9-b2bb-771ec69ae0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:08.388630 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1429/2841] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:43:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:43:08.399928 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181a72a-b75a-461b-9e33-8c6379ad2e33 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:08.401793 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6181a72a-b75a-461b-9e33-8c6379ad2e33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:08.421985 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:43:08.464668 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181a72a-b75a-461b-9e33-8c6379ad2e33 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:08.469051 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcaf551-abf4-4688-b7cb-8a3c08843f99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:08.470106 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcaf551-abf4-4688-b7cb-8a3c08843f99 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:08.470399 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcaf551-abf4-4688-b7cb-8a3c08843f99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:08.470717 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcaf551-abf4-4688-b7cb-8a3c08843f99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:08.480133 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcaf551-abf4-4688-b7cb-8a3c08843f99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:08.482407 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcaf551-abf4-4688-b7cb-8a3c08843f99 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:08.491188 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1413/2842] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:43:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:43:10.773442 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9edcf6-1f80-4be9-b2bb-771ec69ae0c8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:10.789751 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9edcf6-1f80-4be9-b2bb-771ec69ae0c8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:10.789751 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c5374-ae08-4eca-b1d7-cdc707b6fd20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:10.790159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c5374-ae08-4eca-b1d7-cdc707b6fd20 service cinder] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:10.790430 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c5374-ae08-4eca-b1d7-cdc707b6fd20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:10.790802 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c5374-ae08-4eca-b1d7-cdc707b6fd20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:10.794001 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-d83c5374-ae08-4eca-b1d7-cdc707b6fd20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:10.817366 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:43:10.862926 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c5374-ae08-4eca-b1d7-cdc707b6fd20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:10.864447 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c5374-ae08-4eca-b1d7-cdc707b6fd20 service cinder] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:10.874016 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1430/2843] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:43:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:43:18.586343 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=19 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:43:21.341626 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=19 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:43:25.924770 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d878c-4509-43f1-a74b-d65098fae01f None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:25.925288 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d878c-4509-43f1-a74b-d65098fae01f None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:25.925391 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d878c-4509-43f1-a74b-d65098fae01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:25.943581 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-4c4d878c-4509-43f1-a74b-d65098fae01f None None] MFA Rules not processed for user `bb8a8a4e8b634322a92992379c98b721`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:43:25.945234 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4c4d878c-4509-43f1-a74b-d65098fae01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:25.952336 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1414/2844] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:43:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:43:32.317832 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83a687-fe7a-4620-ab23-a1c1159db334 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:32.318414 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83a687-fe7a-4620-ab23-a1c1159db334 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:32.318414 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83a687-fe7a-4620-ab23-a1c1159db334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:32.338167 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-ec83a687-fe7a-4620-ab23-a1c1159db334 None None] MFA Rules not processed for user `d49604bffa3d4e89a631eefa6d4e4f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:43:32.339331 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ec83a687-fe7a-4620-ab23-a1c1159db334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:32.397874 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1431/2845] 10.222.0.67 () {58 vars in 1009 bytes} [Thu Jun 30 16:43:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:43:32.412700 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d878c-4509-43f1-a74b-d65098fae01f None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:32.414305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4c4d878c-4509-43f1-a74b-d65098fae01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:32.489032 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d878c-4509-43f1-a74b-d65098fae01f None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:32.494281 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011d2a5-45b9-49ec-89b1-cab68bbcabd6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988b917eea4b48ffae32e23b8555a5c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:32.494934 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011d2a5-45b9-49ec-89b1-cab68bbcabd6 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:32.495215 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011d2a5-45b9-49ec-89b1-cab68bbcabd6 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:32.495522 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011d2a5-45b9-49ec-89b1-cab68bbcabd6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:32.504630 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8011d2a5-45b9-49ec-89b1-cab68bbcabd6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:32.506210 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8011d2a5-45b9-49ec-89b1-cab68bbcabd6 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:32.514459 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1415/2846] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 30 16:43:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 16:43:34.597190 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83a687-fe7a-4620-ab23-a1c1159db334 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:34.747212 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83a687-fe7a-4620-ab23-a1c1159db334 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:34.761302 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de3a75-74b6-461d-b88e-585d8a186f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:34.764397 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de3a75-74b6-461d-b88e-585d8a186f34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:34.764397 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de3a75-74b6-461d-b88e-585d8a186f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:34.764397 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de3a75-74b6-461d-b88e-585d8a186f34 admin admin] PATH_INFO: `/v3/users/bad13e4e5a134b4fb2611c7aee3b0ba8` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:34.764699 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de3a75-74b6-461d-b88e-585d8a186f34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bad13e4e5a134b4fb2611c7aee3b0ba8)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:34.767599 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de3a75-74b6-461d-b88e-585d8a186f34 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:34.963780 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a1de3a75-74b6-461d-b88e-585d8a186f34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bad13e4e5a134b4fb2611c7aee3b0ba8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:34.966336 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a1de3a75-74b6-461d-b88e-585d8a186f34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bad13e4e5a134b4fb2611c7aee3b0ba8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:34.989400 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a1de3a75-74b6-461d-b88e-585d8a186f34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bad13e4e5a134b4fb2611c7aee3b0ba8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:35.119151 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a1de3a75-74b6-461d-b88e-585d8a186f34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bad13e4e5a134b4fb2611c7aee3b0ba8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:35.120264 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-a1de3a75-74b6-461d-b88e-585d8a186f34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bad13e4e5a134b4fb2611c7aee3b0ba8'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:35.153579 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1432/2847] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:43:34 2022] DELETE /identity/v3/users/bad13e4e5a134b4fb2611c7aee3b0ba8 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:43:35.162734 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017fbaf-6a6f-46cc-aa68-642895ff0f25 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:35.163709 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017fbaf-6a6f-46cc-aa68-642895ff0f25 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:35.164264 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f017fbaf-6a6f-46cc-aa68-642895ff0f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:35.202083 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-f017fbaf-6a6f-46cc-aa68-642895ff0f25 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:43:35.206457 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f017fbaf-6a6f-46cc-aa68-642895ff0f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:35.336321 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1416/2848] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:43:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:43:35.343165 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de3a75-74b6-461d-b88e-585d8a186f34 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:35.345593 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-a1de3a75-74b6-461d-b88e-585d8a186f34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:35.442041 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:43:35.485147 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de3a75-74b6-461d-b88e-585d8a186f34 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:35.490952 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a58cf-6452-499e-8f5b-98d064b34693 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:35.491890 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a58cf-6452-499e-8f5b-98d064b34693 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:35.492226 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a58cf-6452-499e-8f5b-98d064b34693 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:35.492513 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a58cf-6452-499e-8f5b-98d064b34693 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:35.506760 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a58cf-6452-499e-8f5b-98d064b34693 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:35.514426 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a58cf-6452-499e-8f5b-98d064b34693 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:35.538178 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1433/2849] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:43:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:43:36.056832 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017fbaf-6a6f-46cc-aa68-642895ff0f25 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:36.058767 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-f017fbaf-6a6f-46cc-aa68-642895ff0f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:36.122528 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f017fbaf-6a6f-46cc-aa68-642895ff0f25 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:36.135542 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb092a-b17c-49af-94b7-68cf6933d4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:36.136564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb092a-b17c-49af-94b7-68cf6933d4f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:36.136873 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb092a-b17c-49af-94b7-68cf6933d4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:36.137203 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb092a-b17c-49af-94b7-68cf6933d4f0 admin admin] PATH_INFO: `/v3/projects/813b58b12d3f4b7a855d9c6c20e0664e` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:36.138860 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb092a-b17c-49af-94b7-68cf6933d4f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=813b58b12d3f4b7a855d9c6c20e0664e)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:36.140995 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb092a-b17c-49af-94b7-68cf6933d4f0 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:36.357922 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-84bb092a-b17c-49af-94b7-68cf6933d4f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '813b58b12d3f4b7a855d9c6c20e0664e', 'request_id': 'req-84bb092a-b17c-49af-94b7-68cf6933d4f0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:36.364800 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-84bb092a-b17c-49af-94b7-68cf6933d4f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '813b58b12d3f4b7a855d9c6c20e0664e', 'request_id': 'req-84bb092a-b17c-49af-94b7-68cf6933d4f0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:36.370858 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-84bb092a-b17c-49af-94b7-68cf6933d4f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '813b58b12d3f4b7a855d9c6c20e0664e', 'request_id': 'req-84bb092a-b17c-49af-94b7-68cf6933d4f0'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:36.466881 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-84bb092a-b17c-49af-94b7-68cf6933d4f0 admin admin] The token cache is being invalidate because project 813b58b12d3f4b7a855d9c6c20e0664e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:43:36.467173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-84bb092a-b17c-49af-94b7-68cf6933d4f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:36.468998 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-84bb092a-b17c-49af-94b7-68cf6933d4f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:36.474092 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1417/2850] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:43:36 2022] DELETE /identity/v3/projects/813b58b12d3f4b7a855d9c6c20e0664e => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:43:36.476490 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e78d0-0442-4dd7-a86f-6f55fc663b4b None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:36.477418 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e78d0-0442-4dd7-a86f-6f55fc663b4b None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:36.477934 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e78d0-0442-4dd7-a86f-6f55fc663b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:36.499094 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-404e78d0-0442-4dd7-a86f-6f55fc663b4b None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:43:36.503231 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-404e78d0-0442-4dd7-a86f-6f55fc663b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:36.605372 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1434/2851] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:43:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:43:36.614933 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1922bb-1950-4a29-98fd-d7e1e365b339 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:36.615272 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1922bb-1950-4a29-98fd-d7e1e365b339 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:36.617838 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1922bb-1950-4a29-98fd-d7e1e365b339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:36.636067 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:43:36.643133 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-2a1922bb-1950-4a29-98fd-d7e1e365b339 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:43:36.645504 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-2a1922bb-1950-4a29-98fd-d7e1e365b339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:36.657781 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1418/2852] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:43:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:43:36.659833 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e78d0-0442-4dd7-a86f-6f55fc663b4b None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:36.689945 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e78d0-0442-4dd7-a86f-6f55fc663b4b None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:36.696596 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d09da8-4053-4f36-86b4-f3d8968e7982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:36.697462 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d09da8-4053-4f36-86b4-f3d8968e7982 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:36.697900 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d09da8-4053-4f36-86b4-f3d8968e7982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:36.698334 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d09da8-4053-4f36-86b4-f3d8968e7982 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:36.699363 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d09da8-4053-4f36-86b4-f3d8968e7982 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:43:36.701544 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d09da8-4053-4f36-86b4-f3d8968e7982 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:36.703375 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d09da8-4053-4f36-86b4-f3d8968e7982 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:36.720693 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1435/2853] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:43:36.722457 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1922bb-1950-4a29-98fd-d7e1e365b339 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:36.731981 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1922bb-1950-4a29-98fd-d7e1e365b339 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:36.736892 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba928eef-a4a6-4d47-b1f5-a94bb9213eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:36.738156 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba928eef-a4a6-4d47-b1f5-a94bb9213eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:36.738534 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba928eef-a4a6-4d47-b1f5-a94bb9213eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:36.738883 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba928eef-a4a6-4d47-b1f5-a94bb9213eb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:36.739687 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba928eef-a4a6-4d47-b1f5-a94bb9213eb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:36.741707 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba928eef-a4a6-4d47-b1f5-a94bb9213eb7 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:36.810771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1419/2854] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:43:36 2022] POST /identity/v3/projects => generated 397 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:43:36.812389 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d09da8-4053-4f36-86b4-f3d8968e7982 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:36.826138 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d09da8-4053-4f36-86b4-f3d8968e7982 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:36.888335 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd774ed9-89db-4254-b163-318a3e55df25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:36.889059 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd774ed9-89db-4254-b163-318a3e55df25 admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:36.889345 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd774ed9-89db-4254-b163-318a3e55df25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:36.889616 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd774ed9-89db-4254-b163-318a3e55df25 admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:36.890324 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd774ed9-89db-4254-b163-318a3e55df25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:36.892290 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd774ed9-89db-4254-b163-318a3e55df25 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:37.046022 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1436/2855] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:43:36 2022] POST /identity/v3/users => generated 299 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:43:37.050571 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba928eef-a4a6-4d47-b1f5-a94bb9213eb7 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:37.064423 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba928eef-a4a6-4d47-b1f5-a94bb9213eb7 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:37.074217 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a3f14-e16a-44d4-8d26-495da69fc716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:37.076156 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a3f14-e16a-44d4-8d26-495da69fc716 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:37.076156 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a3f14-e16a-44d4-8d26-495da69fc716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:37.076510 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a3f14-e16a-44d4-8d26-495da69fc716 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:37.077425 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a3f14-e16a-44d4-8d26-495da69fc716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:37.079772 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a3f14-e16a-44d4-8d26-495da69fc716 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:37.097738 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1420/2856] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:43:37.099277 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd774ed9-89db-4254-b163-318a3e55df25 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:37.110831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd774ed9-89db-4254-b163-318a3e55df25 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:37.116269 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9142fdbc-eca7-45a9-bd4d-6aabae19cb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:37.117073 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142fdbc-eca7-45a9-bd4d-6aabae19cb1c admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:37.117367 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142fdbc-eca7-45a9-bd4d-6aabae19cb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:37.117659 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142fdbc-eca7-45a9-bd4d-6aabae19cb1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:37.118505 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9142fdbc-eca7-45a9-bd4d-6aabae19cb1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:37.120217 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9142fdbc-eca7-45a9-bd4d-6aabae19cb1c admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:37.132153 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1437/2857] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:43:37.141128 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a3f14-e16a-44d4-8d26-495da69fc716 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:37.149575 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a3f14-e16a-44d4-8d26-495da69fc716 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:37.154559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda7601-946a-4ade-a446-cb28267d9d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:37.155586 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda7601-946a-4ade-a446-cb28267d9d9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:37.155936 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda7601-946a-4ade-a446-cb28267d9d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:37.156341 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda7601-946a-4ade-a446-cb28267d9d9b admin admin] PATH_INFO: `/v3/projects/fdb943f19c714cce97aa4897e11299e2/users/8ac873949acb41d18692671e4e0672c3/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:37.174505 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda7601-946a-4ade-a446-cb28267d9d9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdb943f19c714cce97aa4897e11299e2, user_id=8ac873949acb41d18692671e4e0672c3, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:37.177596 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dda7601-946a-4ade-a446-cb28267d9d9b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:37.279776 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1421/2858] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:43:37 2022] PUT /identity/v3/projects/fdb943f19c714cce97aa4897e11299e2/users/8ac873949acb41d18692671e4e0672c3/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:43:37.281831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb87af-842e-46c0-bffb-2df11f5995e5 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:37.283319 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb87af-842e-46c0-bffb-2df11f5995e5 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:37.284350 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbb87af-842e-46c0-bffb-2df11f5995e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:37.360868 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-2cbb87af-842e-46c0-bffb-2df11f5995e5 None None] MFA Rules not processed for user `8ac873949acb41d18692671e4e0672c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:43:37.362633 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2cbb87af-842e-46c0-bffb-2df11f5995e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:37.408456 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:43:37.490971 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1438/2859] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:43:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:43:37.506302 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda7601-946a-4ade-a446-cb28267d9d9b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:37.513022 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-0dda7601-946a-4ade-a446-cb28267d9d9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:37.584306 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda7601-946a-4ade-a446-cb28267d9d9b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:37.588657 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c849c2-9314-4e4e-bbbb-ab175ef80707 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61437a2fa2a94b35b1b4a5750f6e2cad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:37.589415 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c849c2-9314-4e4e-bbbb-ab175ef80707 service cinder] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:37.589901 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c849c2-9314-4e4e-bbbb-ab175ef80707 service cinder] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:37.590281 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c849c2-9314-4e4e-bbbb-ab175ef80707 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:37.602794 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c849c2-9314-4e4e-bbbb-ab175ef80707 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:37.604479 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c849c2-9314-4e4e-bbbb-ab175ef80707 service cinder] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:37.617634 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1422/2860] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:43:37 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:43:45.589960 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb87af-842e-46c0-bffb-2df11f5995e5 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:45.591501 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-2cbb87af-842e-46c0-bffb-2df11f5995e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:45.657255 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbb87af-842e-46c0-bffb-2df11f5995e5 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:45.661335 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ef7ae-d88f-407e-8a86-e11705277a77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:45.661976 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ef7ae-d88f-407e-8a86-e11705277a77 service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:45.662371 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ef7ae-d88f-407e-8a86-e11705277a77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:45.662617 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ef7ae-d88f-407e-8a86-e11705277a77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:45.664406 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ce3ef7ae-d88f-407e-8a86-e11705277a77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:45.744819 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ef7ae-d88f-407e-8a86-e11705277a77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:45.746607 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ef7ae-d88f-407e-8a86-e11705277a77 service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:45.762919 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1439/2861] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:43:45 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:43:47.023179 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c849c2-9314-4e4e-bbbb-ab175ef80707 service cinder] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:47.120421 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c849c2-9314-4e4e-bbbb-ab175ef80707 service cinder] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:47.127458 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5667f90-4bca-4b49-986d-2a3849ae176b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:47.128926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5667f90-4bca-4b49-986d-2a3849ae176b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:47.129213 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5667f90-4bca-4b49-986d-2a3849ae176b admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:47.129494 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5667f90-4bca-4b49-986d-2a3849ae176b admin admin] PATH_INFO: `/v3/users/8ac873949acb41d18692671e4e0672c3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:47.131330 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5667f90-4bca-4b49-986d-2a3849ae176b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ac873949acb41d18692671e4e0672c3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:47.134261 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5667f90-4bca-4b49-986d-2a3849ae176b admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:47.419177 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b5667f90-4bca-4b49-986d-2a3849ae176b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ac873949acb41d18692671e4e0672c3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:47.421394 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b5667f90-4bca-4b49-986d-2a3849ae176b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ac873949acb41d18692671e4e0672c3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:47.424738 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:43:47.532298 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b5667f90-4bca-4b49-986d-2a3849ae176b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ac873949acb41d18692671e4e0672c3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:47.557841 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b5667f90-4bca-4b49-986d-2a3849ae176b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ac873949acb41d18692671e4e0672c3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:47.558627 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-b5667f90-4bca-4b49-986d-2a3849ae176b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ac873949acb41d18692671e4e0672c3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:47.591137 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1423/2862] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:43:47 2022] DELETE /identity/v3/users/8ac873949acb41d18692671e4e0672c3 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:43:47.592869 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc930b13-acd0-4225-802b-6a8b2f4ac5e8 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:47.593683 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc930b13-acd0-4225-802b-6a8b2f4ac5e8 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:47.594558 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc930b13-acd0-4225-802b-6a8b2f4ac5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:47.615451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fc930b13-acd0-4225-802b-6a8b2f4ac5e8 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:43:47.616628 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fc930b13-acd0-4225-802b-6a8b2f4ac5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:47.711625 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1440/2863] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:43:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:43:47.718926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5667f90-4bca-4b49-986d-2a3849ae176b admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:47.721964 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-b5667f90-4bca-4b49-986d-2a3849ae176b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:47.797099 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5667f90-4bca-4b49-986d-2a3849ae176b admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:47.801953 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f47b18-ffe0-4524-99ac-e25ba36ec290 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:47.805768 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f47b18-ffe0-4524-99ac-e25ba36ec290 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:47.805768 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f47b18-ffe0-4524-99ac-e25ba36ec290 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:47.805768 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f47b18-ffe0-4524-99ac-e25ba36ec290 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:47.822668 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f47b18-ffe0-4524-99ac-e25ba36ec290 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:47.824418 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f47b18-ffe0-4524-99ac-e25ba36ec290 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:47.838528 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1424/2864] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:43:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:43:48.559151 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc930b13-acd0-4225-802b-6a8b2f4ac5e8 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:48.561152 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fc930b13-acd0-4225-802b-6a8b2f4ac5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:48.575189 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc930b13-acd0-4225-802b-6a8b2f4ac5e8 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:48.581199 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506db76b-cecc-4e3d-aeaf-bfff70781528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:48.582534 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506db76b-cecc-4e3d-aeaf-bfff70781528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:48.582931 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506db76b-cecc-4e3d-aeaf-bfff70781528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:48.583452 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506db76b-cecc-4e3d-aeaf-bfff70781528 admin admin] PATH_INFO: `/v3/projects/fdb943f19c714cce97aa4897e11299e2` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:48.585103 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506db76b-cecc-4e3d-aeaf-bfff70781528 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdb943f19c714cce97aa4897e11299e2)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:48.587388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506db76b-cecc-4e3d-aeaf-bfff70781528 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:48.604195 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=100 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:43:48.678705 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f47b18-ffe0-4524-99ac-e25ba36ec290 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:48.695191 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f47b18-ffe0-4524-99ac-e25ba36ec290 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:48.701899 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a972f38-c2d2-4ede-ba64-478c460008d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:48.702581 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a972f38-c2d2-4ede-ba64-478c460008d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:48.702884 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a972f38-c2d2-4ede-ba64-478c460008d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:48.703242 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a972f38-c2d2-4ede-ba64-478c460008d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:48.715267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8a972f38-c2d2-4ede-ba64-478c460008d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:48.744551 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-506db76b-cecc-4e3d-aeaf-bfff70781528 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdb943f19c714cce97aa4897e11299e2', 'request_id': 'req-506db76b-cecc-4e3d-aeaf-bfff70781528'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:48.833086 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-506db76b-cecc-4e3d-aeaf-bfff70781528 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdb943f19c714cce97aa4897e11299e2', 'request_id': 'req-506db76b-cecc-4e3d-aeaf-bfff70781528'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:48.841220 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-506db76b-cecc-4e3d-aeaf-bfff70781528 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdb943f19c714cce97aa4897e11299e2', 'request_id': 'req-506db76b-cecc-4e3d-aeaf-bfff70781528'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:48.874742 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-506db76b-cecc-4e3d-aeaf-bfff70781528 admin admin] The token cache is being invalidate because project fdb943f19c714cce97aa4897e11299e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:43:48.875691 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-506db76b-cecc-4e3d-aeaf-bfff70781528 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:48.875832 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-506db76b-cecc-4e3d-aeaf-bfff70781528 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:48.878059 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1441/2865] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:43:48 2022] DELETE /identity/v3/projects/fdb943f19c714cce97aa4897e11299e2 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:43:48.954638 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a972f38-c2d2-4ede-ba64-478c460008d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:48.956985 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a972f38-c2d2-4ede-ba64-478c460008d5 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:48.979373 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1425/2866] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:43:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:43:50.296190 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506db76b-cecc-4e3d-aeaf-bfff70781528 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:50.298347 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-506db76b-cecc-4e3d-aeaf-bfff70781528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:50.367701 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506db76b-cecc-4e3d-aeaf-bfff70781528 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:50.372302 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40909c-e5ed-43a3-96d1-873b2eda0bbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:50.372981 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40909c-e5ed-43a3-96d1-873b2eda0bbf service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:50.373638 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40909c-e5ed-43a3-96d1-873b2eda0bbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:50.374036 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40909c-e5ed-43a3-96d1-873b2eda0bbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:50.377090 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-5e40909c-e5ed-43a3-96d1-873b2eda0bbf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:50.473159 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40909c-e5ed-43a3-96d1-873b2eda0bbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:50.474643 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40909c-e5ed-43a3-96d1-873b2eda0bbf service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:50.485801 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1442/2867] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:43:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:43:50.815738 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a972f38-c2d2-4ede-ba64-478c460008d5 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:50.834313 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a972f38-c2d2-4ede-ba64-478c460008d5 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:50.834313 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49cf6e-669c-413c-a756-a1e688ce3685 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:50.834313 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49cf6e-669c-413c-a756-a1e688ce3685 service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:50.834313 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49cf6e-669c-413c-a756-a1e688ce3685 service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:50.834728 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b49cf6e-669c-413c-a756-a1e688ce3685 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:50.834917 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6b49cf6e-669c-413c-a756-a1e688ce3685 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:50.885459 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:43:50.901559 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49cf6e-669c-413c-a756-a1e688ce3685 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:50.912836 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b49cf6e-669c-413c-a756-a1e688ce3685 service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:50.914270 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1426/2868] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:43:50 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:43:51.934656 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafadb3b-72a3-4b25-a341-b09a44c476b4 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:51.935156 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafadb3b-72a3-4b25-a341-b09a44c476b4 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:51.935407 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafadb3b-72a3-4b25-a341-b09a44c476b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:51.958671 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-fafadb3b-72a3-4b25-a341-b09a44c476b4 None None] MFA Rules not processed for user `d49604bffa3d4e89a631eefa6d4e4f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:43:51.960047 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fafadb3b-72a3-4b25-a341-b09a44c476b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:51.998620 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:43:52.026148 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1443/2869] 10.222.0.67 () {58 vars in 967 bytes} [Thu Jun 30 16:43:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:43:52.050059 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49cf6e-669c-413c-a756-a1e688ce3685 service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:52.050059 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6b49cf6e-669c-413c-a756-a1e688ce3685 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:52.145740 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b49cf6e-669c-413c-a756-a1e688ce3685 service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:52.151491 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0647f-754e-4c1c-832a-7536c6760895 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988b917eea4b48ffae32e23b8555a5c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:52.152176 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0647f-754e-4c1c-832a-7536c6760895 service swift] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:52.152461 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0647f-754e-4c1c-832a-7536c6760895 service swift] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:52.152799 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c0647f-754e-4c1c-832a-7536c6760895 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:52.163704 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0647f-754e-4c1c-832a-7536c6760895 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:52.167891 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c0647f-754e-4c1c-832a-7536c6760895 service swift] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:52.176290 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1427/2870] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 30 16:43:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 16:43:55.606899 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafadb3b-72a3-4b25-a341-b09a44c476b4 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:55.608427 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fafadb3b-72a3-4b25-a341-b09a44c476b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:55.678904 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafadb3b-72a3-4b25-a341-b09a44c476b4 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:55.683557 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42836326-dc3d-408f-b29d-f27fffa1c6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:55.684441 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42836326-dc3d-408f-b29d-f27fffa1c6cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:55.684728 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42836326-dc3d-408f-b29d-f27fffa1c6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:55.685041 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42836326-dc3d-408f-b29d-f27fffa1c6cb admin admin] PATH_INFO: `/v3/users/f58941916ebe4a68a9a9a2b04b745ee2` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:55.686274 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42836326-dc3d-408f-b29d-f27fffa1c6cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f58941916ebe4a68a9a9a2b04b745ee2)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:55.688312 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42836326-dc3d-408f-b29d-f27fffa1c6cb admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:55.889772 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-42836326-dc3d-408f-b29d-f27fffa1c6cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f58941916ebe4a68a9a9a2b04b745ee2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:55.890451 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-42836326-dc3d-408f-b29d-f27fffa1c6cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f58941916ebe4a68a9a9a2b04b745ee2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:55.898282 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-42836326-dc3d-408f-b29d-f27fffa1c6cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f58941916ebe4a68a9a9a2b04b745ee2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:55.969603 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-42836326-dc3d-408f-b29d-f27fffa1c6cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f58941916ebe4a68a9a9a2b04b745ee2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:55.970419 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-42836326-dc3d-408f-b29d-f27fffa1c6cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f58941916ebe4a68a9a9a2b04b745ee2'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:55.992479 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1444/2871] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:43:55 2022] DELETE /identity/v3/users/f58941916ebe4a68a9a9a2b04b745ee2 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:43:56.263313 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0647f-754e-4c1c-832a-7536c6760895 service swift] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:56.277297 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c0647f-754e-4c1c-832a-7536c6760895 service swift] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:56.344734 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e461b451-23cb-4716-9a7f-bae8c18c2815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:56.345602 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461b451-23cb-4716-9a7f-bae8c18c2815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:56.346171 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461b451-23cb-4716-9a7f-bae8c18c2815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:56.346463 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e461b451-23cb-4716-9a7f-bae8c18c2815 admin admin] PATH_INFO: `/v3/projects/fc48c4912c8647d482593a28474d6b14` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:56.347715 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461b451-23cb-4716-9a7f-bae8c18c2815 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc48c4912c8647d482593a28474d6b14)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:56.349629 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e461b451-23cb-4716-9a7f-bae8c18c2815 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:56.457742 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e461b451-23cb-4716-9a7f-bae8c18c2815 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc48c4912c8647d482593a28474d6b14', 'request_id': 'req-e461b451-23cb-4716-9a7f-bae8c18c2815'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:56.463879 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e461b451-23cb-4716-9a7f-bae8c18c2815 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc48c4912c8647d482593a28474d6b14', 'request_id': 'req-e461b451-23cb-4716-9a7f-bae8c18c2815'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:56.469485 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e461b451-23cb-4716-9a7f-bae8c18c2815 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc48c4912c8647d482593a28474d6b14', 'request_id': 'req-e461b451-23cb-4716-9a7f-bae8c18c2815'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:56.558132 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e461b451-23cb-4716-9a7f-bae8c18c2815 admin admin] The token cache is being invalidate because project fc48c4912c8647d482593a28474d6b14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:43:56.558424 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e461b451-23cb-4716-9a7f-bae8c18c2815 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:56.559048 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-e461b451-23cb-4716-9a7f-bae8c18c2815 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:43:56.560631 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1428/2872] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:43:56 2022] DELETE /identity/v3/projects/fc48c4912c8647d482593a28474d6b14 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:43:56.566164 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8e402-e90d-4a99-8cca-72b0808912c0 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:56.566459 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8e402-e90d-4a99-8cca-72b0808912c0 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:56.566743 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8e402-e90d-4a99-8cca-72b0808912c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:56.585397 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-3cb8e402-e90d-4a99-8cca-72b0808912c0 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:43:56.586606 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3cb8e402-e90d-4a99-8cca-72b0808912c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:56.651654 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1445/2873] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:43:56.659829 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afd319-f063-4cff-8014-1dc625a1f3b4 None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:56.660119 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afd319-f063-4cff-8014-1dc625a1f3b4 None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:56.660398 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40afd319-f063-4cff-8014-1dc625a1f3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:56.673028 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:43:56.680152 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-40afd319-f063-4cff-8014-1dc625a1f3b4 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:43:56.681369 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-40afd319-f063-4cff-8014-1dc625a1f3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:56.689647 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1429/2874] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:43:56.695430 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb8e402-e90d-4a99-8cca-72b0808912c0 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:56.700425 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:43:56.706115 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb8e402-e90d-4a99-8cca-72b0808912c0 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:56.710892 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f762d-4aa9-4441-9bce-5c6636f2de88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:56.711575 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f762d-4aa9-4441-9bce-5c6636f2de88 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:56.711863 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f762d-4aa9-4441-9bce-5c6636f2de88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:56.712162 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f762d-4aa9-4441-9bce-5c6636f2de88 admin admin] PATH_INFO: `/v3/domains` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:56.717203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f762d-4aa9-4441-9bce-5c6636f2de88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80298) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 16:43:56.717203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f762d-4aa9-4441-9bce-5c6636f2de88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:56.717203 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f762d-4aa9-4441-9bce-5c6636f2de88 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:56.726865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1446/2875] 10.222.0.67 () {58 vars in 1152 bytes} [Thu Jun 30 16:43:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 16:43:56.731742 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afd319-f063-4cff-8014-1dc625a1f3b4 None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:56.741756 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40afd319-f063-4cff-8014-1dc625a1f3b4 None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:56.746530 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58ce8b-e8c7-44c5-9fcd-70d1e30f053d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:56.751581 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58ce8b-e8c7-44c5-9fcd-70d1e30f053d admin admin] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:56.751581 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58ce8b-e8c7-44c5-9fcd-70d1e30f053d admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:56.751581 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b58ce8b-e8c7-44c5-9fcd-70d1e30f053d admin admin] PATH_INFO: `/v3/projects` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:56.751581 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b58ce8b-e8c7-44c5-9fcd-70d1e30f053d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:56.751581 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b58ce8b-e8c7-44c5-9fcd-70d1e30f053d admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:56.880959 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1430/2876] 10.222.0.67 () {60 vars in 1152 bytes} [Thu Jun 30 16:43:56 2022] POST /identity/v3/projects => generated 367 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 16:43:56.885490 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f762d-4aa9-4441-9bce-5c6636f2de88 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:56.895823 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f762d-4aa9-4441-9bce-5c6636f2de88 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:56.955241 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6731569-e28f-48e2-aa5d-597a3dea8d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:56.955943 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6731569-e28f-48e2-aa5d-597a3dea8d9d admin admin] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:56.956233 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6731569-e28f-48e2-aa5d-597a3dea8d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:56.956588 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6731569-e28f-48e2-aa5d-597a3dea8d9d admin admin] PATH_INFO: `/v3/users` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:56.957287 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6731569-e28f-48e2-aa5d-597a3dea8d9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:56.959315 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6731569-e28f-48e2-aa5d-597a3dea8d9d admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:57.076359 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1447/2877] 10.222.0.67 () {60 vars in 1143 bytes} [Thu Jun 30 16:43:56 2022] POST /identity/v3/users => generated 284 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 16:43:57.080768 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58ce8b-e8c7-44c5-9fcd-70d1e30f053d admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:57.091942 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b58ce8b-e8c7-44c5-9fcd-70d1e30f053d admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:57.096477 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28faf90-d157-42e7-b068-28ca57482384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:57.097634 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28faf90-d157-42e7-b068-28ca57482384 admin admin] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:57.098502 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28faf90-d157-42e7-b068-28ca57482384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:57.098778 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28faf90-d157-42e7-b068-28ca57482384 admin admin] PATH_INFO: `/v3/roles` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:57.099563 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28faf90-d157-42e7-b068-28ca57482384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:57.101147 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28faf90-d157-42e7-b068-28ca57482384 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:57.111813 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1431/2878] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:43:57.116937 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6731569-e28f-48e2-aa5d-597a3dea8d9d admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:57.127062 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6731569-e28f-48e2-aa5d-597a3dea8d9d admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:57.132415 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52c691c-77f8-457b-be38-9f946ff42df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:57.134377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c691c-77f8-457b-be38-9f946ff42df0 admin admin] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:57.134377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c691c-77f8-457b-be38-9f946ff42df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:57.134377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52c691c-77f8-457b-be38-9f946ff42df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:57.134377 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c691c-77f8-457b-be38-9f946ff42df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:57.135927 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52c691c-77f8-457b-be38-9f946ff42df0 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:57.146383 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1448/2879] 10.222.0.67 () {58 vars in 1121 bytes} [Thu Jun 30 16:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 16:43:57.150693 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28faf90-d157-42e7-b068-28ca57482384 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:57.161577 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28faf90-d157-42e7-b068-28ca57482384 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:57.166524 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb1a16c-0cbc-48ca-9b7a-14754ba337fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:57.167421 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1a16c-0cbc-48ca-9b7a-14754ba337fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:57.167739 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1a16c-0cbc-48ca-9b7a-14754ba337fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:57.168043 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1a16c-0cbc-48ca-9b7a-14754ba337fe admin admin] PATH_INFO: `/v3/projects/c8ab9d44217d440dbb99b18c5a4ab2c0/users/e4377f0d83e84342a1e4fae8880424b3/roles/9ab3e302ead54950b7c30103f1e7fda7` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:57.184078 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb1a16c-0cbc-48ca-9b7a-14754ba337fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8ab9d44217d440dbb99b18c5a4ab2c0, user_id=e4377f0d83e84342a1e4fae8880424b3, role_id=9ab3e302ead54950b7c30103f1e7fda7)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:57.186738 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb1a16c-0cbc-48ca-9b7a-14754ba337fe admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:57.293455 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1432/2880] 10.222.0.67 () {60 vars in 1482 bytes} [Thu Jun 30 16:43:57 2022] PUT /identity/v3/projects/c8ab9d44217d440dbb99b18c5a4ab2c0/users/e4377f0d83e84342a1e4fae8880424b3/roles/9ab3e302ead54950b7c30103f1e7fda7 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:43:57.304617 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e31b6-8619-4044-8258-cd6475c4ecfa None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:57.304913 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e31b6-8619-4044-8258-cd6475c4ecfa None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:57.305200 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e31b6-8619-4044-8258-cd6475c4ecfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:57.332989 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-699e31b6-8619-4044-8258-cd6475c4ecfa None None] MFA Rules not processed for user `e4377f0d83e84342a1e4fae8880424b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:43:57.334292 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-699e31b6-8619-4044-8258-cd6475c4ecfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:57.399348 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1449/2881] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:43:57 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:43:57.412302 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb1a16c-0cbc-48ca-9b7a-14754ba337fe admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:43:57.413991 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-dbb1a16c-0cbc-48ca-9b7a-14754ba337fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:43:57.473235 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb1a16c-0cbc-48ca-9b7a-14754ba337fe admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:43:57.477058 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a971f55-716c-414a-a0df-4bdae56ab357 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec41c84c9375437590c42621ca5b400a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:43:57.477702 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a971f55-716c-414a-a0df-4bdae56ab357 service glance] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:43:57.477985 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a971f55-716c-414a-a0df-4bdae56ab357 service glance] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:43:57.478260 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a971f55-716c-414a-a0df-4bdae56ab357 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:43:57.486981 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a971f55-716c-414a-a0df-4bdae56ab357 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:43:57.489111 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a971f55-716c-414a-a0df-4bdae56ab357 service glance] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:43:57.500587 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1433/2882] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:43:57 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 16:44:06.665110 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e31b6-8619-4044-8258-cd6475c4ecfa None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:44:06.666817 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-699e31b6-8619-4044-8258-cd6475c4ecfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:44:06.729224 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e31b6-8619-4044-8258-cd6475c4ecfa None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:44:06.732895 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060fc713-f460-426e-bf5a-05e4b3857d8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:44:06.733533 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fc713-f460-426e-bf5a-05e4b3857d8c service neutron] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:44:06.733823 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fc713-f460-426e-bf5a-05e4b3857d8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:44:06.734097 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fc713-f460-426e-bf5a-05e4b3857d8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:44:06.735926 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-060fc713-f460-426e-bf5a-05e4b3857d8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:44:06.747073 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060fc713-f460-426e-bf5a-05e4b3857d8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:44:06.748575 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060fc713-f460-426e-bf5a-05e4b3857d8c service neutron] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:44:06.753121 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:44:06.757627 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1450/2883] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:44:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:44:07.494342 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=50,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:44:33.379651 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a971f55-716c-414a-a0df-4bdae56ab357 service glance] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:44:33.441227 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a971f55-716c-414a-a0df-4bdae56ab357 service glance] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:44:33.446066 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d522f-304c-49e6-b88a-08cdb52b63d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:44:33.446922 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d522f-304c-49e6-b88a-08cdb52b63d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:44:33.447201 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d522f-304c-49e6-b88a-08cdb52b63d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:44:33.447493 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d522f-304c-49e6-b88a-08cdb52b63d5 admin admin] PATH_INFO: `/v3/users/e4377f0d83e84342a1e4fae8880424b3` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:44:33.448759 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716d522f-304c-49e6-b88a-08cdb52b63d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4377f0d83e84342a1e4fae8880424b3)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:44:33.450666 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716d522f-304c-49e6-b88a-08cdb52b63d5 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:44:33.673284 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-716d522f-304c-49e6-b88a-08cdb52b63d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4377f0d83e84342a1e4fae8880424b3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:44:33.673869 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-716d522f-304c-49e6-b88a-08cdb52b63d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4377f0d83e84342a1e4fae8880424b3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:44:33.755034 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-716d522f-304c-49e6-b88a-08cdb52b63d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4377f0d83e84342a1e4fae8880424b3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:44:33.762724 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-716d522f-304c-49e6-b88a-08cdb52b63d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4377f0d83e84342a1e4fae8880424b3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:44:33.763367 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-716d522f-304c-49e6-b88a-08cdb52b63d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4377f0d83e84342a1e4fae8880424b3'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:44:33.781121 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1434/2884] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:44:33 2022] DELETE /identity/v3/users/e4377f0d83e84342a1e4fae8880424b3 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:44:33.785875 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c94a94-b945-4122-899c-bcea3c0d2273 None None] REQUEST_METHOD: `POST` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:44:33.786420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c94a94-b945-4122-899c-bcea3c0d2273 None None] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:44:33.786420 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c94a94-b945-4122-899c-bcea3c0d2273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:44:33.804452 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.auth.core [None req-13c94a94-b945-4122-899c-bcea3c0d2273 None None] MFA Rules not processed for user `9242c0a59f6b49568f911aaa0ef688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80298) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:44:33.805641 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-13c94a94-b945-4122-899c-bcea3c0d2273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:44:33.866332 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1451/2885] 10.222.0.67 () {58 vars in 957 bytes} [Thu Jun 30 16:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 16:44:33.876293 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d522f-304c-49e6-b88a-08cdb52b63d5 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:44:33.878334 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-716d522f-304c-49e6-b88a-08cdb52b63d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:44:33.936475 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d522f-304c-49e6-b88a-08cdb52b63d5 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:44:33.940620 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f579cb1-9062-4aeb-b4ab-ae260052115b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:44:33.941269 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f579cb1-9062-4aeb-b4ab-ae260052115b service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:44:33.941547 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f579cb1-9062-4aeb-b4ab-ae260052115b service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:44:33.941824 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f579cb1-9062-4aeb-b4ab-ae260052115b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:44:33.951460 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f579cb1-9062-4aeb-b4ab-ae260052115b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:44:33.952904 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f579cb1-9062-4aeb-b4ab-ae260052115b service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:44:33.962564 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1435/2886] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:44:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:44:34.414991 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c94a94-b945-4122-899c-bcea3c0d2273 None None] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:44:34.416581 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-13c94a94-b945-4122-899c-bcea3c0d2273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:44:34.428613 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c94a94-b945-4122-899c-bcea3c0d2273 None None] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:44:34.434089 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34ce87-5531-4d14-a206-2d251dcc32f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:44:34.438625 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34ce87-5531-4d14-a206-2d251dcc32f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:44:34.438907 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34ce87-5531-4d14-a206-2d251dcc32f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:44:34.439183 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34ce87-5531-4d14-a206-2d251dcc32f1 admin admin] PATH_INFO: `/v3/projects/c8ab9d44217d440dbb99b18c5a4ab2c0` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:44:34.441020 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed34ce87-5531-4d14-a206-2d251dcc32f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8ab9d44217d440dbb99b18c5a4ab2c0)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:44:34.442960 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed34ce87-5531-4d14-a206-2d251dcc32f1 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:44:34.578735 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ed34ce87-5531-4d14-a206-2d251dcc32f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8ab9d44217d440dbb99b18c5a4ab2c0', 'request_id': 'req-ed34ce87-5531-4d14-a206-2d251dcc32f1'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:44:34.661234 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ed34ce87-5531-4d14-a206-2d251dcc32f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8ab9d44217d440dbb99b18c5a4ab2c0', 'request_id': 'req-ed34ce87-5531-4d14-a206-2d251dcc32f1'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:44:34.668297 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ed34ce87-5531-4d14-a206-2d251dcc32f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8ab9d44217d440dbb99b18c5a4ab2c0', 'request_id': 'req-ed34ce87-5531-4d14-a206-2d251dcc32f1'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:44:34.679325 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ed34ce87-5531-4d14-a206-2d251dcc32f1 admin admin] The token cache is being invalidate because project c8ab9d44217d440dbb99b18c5a4ab2c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:44:34.679644 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ed34ce87-5531-4d14-a206-2d251dcc32f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:44:34.680300 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ed34ce87-5531-4d14-a206-2d251dcc32f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:44:34.681853 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1452/2887] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:44:34 2022] DELETE /identity/v3/projects/c8ab9d44217d440dbb99b18c5a4ab2c0 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:44:43.958329 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:44:44.676132 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:44:58.630898 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f579cb1-9062-4aeb-b4ab-ae260052115b service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:44:58.630898 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-4f579cb1-9062-4aeb-b4ab-ae260052115b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:44:58.753429 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f579cb1-9062-4aeb-b4ab-ae260052115b service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:44:58.843817 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d225d-38b5-4d8e-a694-b0226464234e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:44:58.850717 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d225d-38b5-4d8e-a694-b0226464234e service nova] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:44:58.850717 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d225d-38b5-4d8e-a694-b0226464234e service nova] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:44:58.850717 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d225d-38b5-4d8e-a694-b0226464234e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:44:58.850717 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6f9d225d-38b5-4d8e-a694-b0226464234e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:44:58.859993 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d225d-38b5-4d8e-a694-b0226464234e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:44:58.862308 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d225d-38b5-4d8e-a694-b0226464234e service nova] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:44:58.874918 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1436/2888] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:44:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:45:05.702123 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34ce87-5531-4d14-a206-2d251dcc32f1 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:45:05.703010 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ed34ce87-5531-4d14-a206-2d251dcc32f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:45:05.720516 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34ce87-5531-4d14-a206-2d251dcc32f1 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:45:05.726669 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521556ba-5c66-47c2-9881-a81a7490ae7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71cca3708386459197abf6c67505bc5f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:45:05.727381 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521556ba-5c66-47c2-9881-a81a7490ae7d service nova] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:45:05.727721 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521556ba-5c66-47c2-9881-a81a7490ae7d service nova] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:45:05.728002 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521556ba-5c66-47c2-9881-a81a7490ae7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:45:05.730079 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-521556ba-5c66-47c2-9881-a81a7490ae7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:45:05.743517 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521556ba-5c66-47c2-9881-a81a7490ae7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:45:05.745144 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521556ba-5c66-47c2-9881-a81a7490ae7d service nova] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:45:05.757960 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1453/2889] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:45:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:45:07.970305 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02a311-1292-43f1-bbf8-0e46070dc1ce None None] REQUEST_METHOD: `POST` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:45:07.970800 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02a311-1292-43f1-bbf8-0e46070dc1ce None None] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:45:07.970926 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a02a311-1292-43f1-bbf8-0e46070dc1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:45:07.990643 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.auth.core [None req-6a02a311-1292-43f1-bbf8-0e46070dc1ce None None] MFA Rules not processed for user `d49604bffa3d4e89a631eefa6d4e4f5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80297) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 16:45:07.994024 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6a02a311-1292-43f1-bbf8-0e46070dc1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:45:08.048493 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1437/2890] 10.222.0.67 () {58 vars in 967 bytes} [Thu Jun 30 16:45:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 16:45:08.187212 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521556ba-5c66-47c2-9881-a81a7490ae7d service nova] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:45:08.188700 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-521556ba-5c66-47c2-9881-a81a7490ae7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:45:08.258477 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521556ba-5c66-47c2-9881-a81a7490ae7d service nova] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:45:08.263730 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c953539-25dd-449e-92e5-4fcf2ec71c6d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '988b917eea4b48ffae32e23b8555a5c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:45:08.264555 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c953539-25dd-449e-92e5-4fcf2ec71c6d service swift] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:45:08.265051 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c953539-25dd-449e-92e5-4fcf2ec71c6d service swift] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:45:08.265359 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c953539-25dd-449e-92e5-4fcf2ec71c6d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:45:08.274831 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c953539-25dd-449e-92e5-4fcf2ec71c6d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:45:08.276500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c953539-25dd-449e-92e5-4fcf2ec71c6d service swift] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:45:08.284827 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1454/2891] 10.222.0.67 () {60 vars in 1384 bytes} [Thu Jun 30 16:45:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 16:45:16.716340 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02a311-1292-43f1-bbf8-0e46070dc1ce None None] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:45:16.718018 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-6a02a311-1292-43f1-bbf8-0e46070dc1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:45:16.752314 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:45:16.785741 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a02a311-1292-43f1-bbf8-0e46070dc1ce None None] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:45:16.791108 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef0e730-fb2d-4815-b5cb-3a33a923c604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:45:16.792008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0e730-fb2d-4815-b5cb-3a33a923c604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:45:16.792294 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0e730-fb2d-4815-b5cb-3a33a923c604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:45:16.792573 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0e730-fb2d-4815-b5cb-3a33a923c604 admin admin] PATH_INFO: `/v3/users/9c4e7816dc9c4eeab652534160c22757` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:45:16.793759 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef0e730-fb2d-4815-b5cb-3a33a923c604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c4e7816dc9c4eeab652534160c22757)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:45:16.795695 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef0e730-fb2d-4815-b5cb-3a33a923c604 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:45:17.017703 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8ef0e730-fb2d-4815-b5cb-3a33a923c604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c4e7816dc9c4eeab652534160c22757'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:17.018362 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8ef0e730-fb2d-4815-b5cb-3a33a923c604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c4e7816dc9c4eeab652534160c22757'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:17.118337 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8ef0e730-fb2d-4815-b5cb-3a33a923c604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c4e7816dc9c4eeab652534160c22757'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:17.126022 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8ef0e730-fb2d-4815-b5cb-3a33a923c604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c4e7816dc9c4eeab652534160c22757'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:17.129061 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-8ef0e730-fb2d-4815-b5cb-3a33a923c604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c4e7816dc9c4eeab652534160c22757'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:17.145383 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1438/2892] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:45:16 2022] DELETE /identity/v3/users/9c4e7816dc9c4eeab652534160c22757 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 16:45:17.430571 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c953539-25dd-449e-92e5-4fcf2ec71c6d service swift] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:45:17.442861 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c953539-25dd-449e-92e5-4fcf2ec71c6d service swift] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:45:17.504530 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbace338-92d8-4424-878b-184b6b46a769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:45:17.505324 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbace338-92d8-4424-878b-184b6b46a769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:45:17.505636 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbace338-92d8-4424-878b-184b6b46a769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:45:17.505914 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbace338-92d8-4424-878b-184b6b46a769 admin admin] PATH_INFO: `/v3/projects/ce9b910eae974562929f660ba8c7af4c` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:45:17.507865 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbace338-92d8-4424-878b-184b6b46a769 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce9b910eae974562929f660ba8c7af4c)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:45:17.509802 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbace338-92d8-4424-878b-184b6b46a769 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:45:17.656957 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cbace338-92d8-4424-878b-184b6b46a769 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce9b910eae974562929f660ba8c7af4c', 'request_id': 'req-cbace338-92d8-4424-878b-184b6b46a769'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:17.752922 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cbace338-92d8-4424-878b-184b6b46a769 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce9b910eae974562929f660ba8c7af4c', 'request_id': 'req-cbace338-92d8-4424-878b-184b6b46a769'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:17.760448 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cbace338-92d8-4424-878b-184b6b46a769 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce9b910eae974562929f660ba8c7af4c', 'request_id': 'req-cbace338-92d8-4424-878b-184b6b46a769'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:17.771007 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cbace338-92d8-4424-878b-184b6b46a769 admin admin] The token cache is being invalidate because project ce9b910eae974562929f660ba8c7af4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:45:17.771859 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cbace338-92d8-4424-878b-184b6b46a769 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:17.772054 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-cbace338-92d8-4424-878b-184b6b46a769 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:17.773650 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1455/2893] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:45:17 2022] DELETE /identity/v3/projects/ce9b910eae974562929f660ba8c7af4c => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:45:17.777576 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef0e730-fb2d-4815-b5cb-3a33a923c604 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:45:17.779052 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-8ef0e730-fb2d-4815-b5cb-3a33a923c604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:45:17.792959 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef0e730-fb2d-4815-b5cb-3a33a923c604 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:45:17.850624 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e499d-84c6-4cd6-b513-4a19a5fa9a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:45:17.851634 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e499d-84c6-4cd6-b513-4a19a5fa9a19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:45:17.851912 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e499d-84c6-4cd6-b513-4a19a5fa9a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:45:17.852183 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e499d-84c6-4cd6-b513-4a19a5fa9a19 admin admin] PATH_INFO: `/v3/users/631ba6e9064243a58465648949c50820` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:45:17.853373 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e499d-84c6-4cd6-b513-4a19a5fa9a19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=631ba6e9064243a58465648949c50820)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:45:17.855276 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e499d-84c6-4cd6-b513-4a19a5fa9a19 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:45:18.077811 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5d7e499d-84c6-4cd6-b513-4a19a5fa9a19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '631ba6e9064243a58465648949c50820'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:18.078475 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5d7e499d-84c6-4cd6-b513-4a19a5fa9a19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '631ba6e9064243a58465648949c50820'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:18.081007 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:45:18.209975 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5d7e499d-84c6-4cd6-b513-4a19a5fa9a19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '631ba6e9064243a58465648949c50820'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:18.217507 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5d7e499d-84c6-4cd6-b513-4a19a5fa9a19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '631ba6e9064243a58465648949c50820'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:18.218173 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-5d7e499d-84c6-4cd6-b513-4a19a5fa9a19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '631ba6e9064243a58465648949c50820'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:18.236752 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1439/2894] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:45:17 2022] DELETE /identity/v3/users/631ba6e9064243a58465648949c50820 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:45:18.493006 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbace338-92d8-4424-878b-184b6b46a769 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:45:18.495842 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-cbace338-92d8-4424-878b-184b6b46a769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:45:18.508143 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbace338-92d8-4424-878b-184b6b46a769 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:45:18.512043 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:45:18.565719 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32fc70-448f-4ce9-988e-403250205acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:45:18.566594 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32fc70-448f-4ce9-988e-403250205acd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:45:18.566884 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32fc70-448f-4ce9-988e-403250205acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:45:18.567166 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32fc70-448f-4ce9-988e-403250205acd admin admin] PATH_INFO: `/v3/projects/00671e957c8644e6954a41aa309e8608` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:45:18.568537 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32fc70-448f-4ce9-988e-403250205acd admin admin] RBAC: Authorizing `identity:delete_project(project_id=00671e957c8644e6954a41aa309e8608)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:45:18.570990 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32fc70-448f-4ce9-988e-403250205acd admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:45:18.702301 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ab32fc70-448f-4ce9-988e-403250205acd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00671e957c8644e6954a41aa309e8608', 'request_id': 'req-ab32fc70-448f-4ce9-988e-403250205acd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:18.771730 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ab32fc70-448f-4ce9-988e-403250205acd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00671e957c8644e6954a41aa309e8608', 'request_id': 'req-ab32fc70-448f-4ce9-988e-403250205acd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:18.779295 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ab32fc70-448f-4ce9-988e-403250205acd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00671e957c8644e6954a41aa309e8608', 'request_id': 'req-ab32fc70-448f-4ce9-988e-403250205acd'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:18.790041 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ab32fc70-448f-4ce9-988e-403250205acd admin admin] The token cache is being invalidate because project 00671e957c8644e6954a41aa309e8608 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80298) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:45:18.790331 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ab32fc70-448f-4ce9-988e-403250205acd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:18.790985 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-ab32fc70-448f-4ce9-988e-403250205acd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:18.792586 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1456/2895] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:45:18 2022] DELETE /identity/v3/projects/00671e957c8644e6954a41aa309e8608 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:45:28.657743 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:DELETE=3,keystone:SELECT=12 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:45:28.863598 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:45:39.120276 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e499d-84c6-4cd6-b513-4a19a5fa9a19 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:45:39.121784 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-5d7e499d-84c6-4cd6-b513-4a19a5fa9a19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:45:39.179913 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e499d-84c6-4cd6-b513-4a19a5fa9a19 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:45:39.231789 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98991a00-afe8-42cb-afda-cacf8b65295e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b508dd8b7244d5f979e1ce7787d2c67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:45:39.232522 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98991a00-afe8-42cb-afda-cacf8b65295e service neutron] REQUEST_METHOD: `GET` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:45:39.232837 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98991a00-afe8-42cb-afda-cacf8b65295e service neutron] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:45:39.233117 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98991a00-afe8-42cb-afda-cacf8b65295e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:45:39.234959 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-98991a00-afe8-42cb-afda-cacf8b65295e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:45:39.309975 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98991a00-afe8-42cb-afda-cacf8b65295e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:45:39.312521 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98991a00-afe8-42cb-afda-cacf8b65295e service neutron] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:45:39.324147 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1440/2896] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:45:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:45:46.738303 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32fc70-448f-4ce9-988e-403250205acd admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:45:46.739955 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-ab32fc70-448f-4ce9-988e-403250205acd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:45:46.825589 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32fc70-448f-4ce9-988e-403250205acd admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:45:46.834779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee7288-70b2-436d-8db8-30ded26f9783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:45:46.834779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee7288-70b2-436d-8db8-30ded26f9783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:45:46.834779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee7288-70b2-436d-8db8-30ded26f9783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:45:46.834779 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee7288-70b2-436d-8db8-30ded26f9783 admin admin] PATH_INFO: `/v3/users/bb8a8a4e8b634322a92992379c98b721` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:45:46.835224 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee7288-70b2-436d-8db8-30ded26f9783 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb8a8a4e8b634322a92992379c98b721)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:45:46.835224 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee7288-70b2-436d-8db8-30ded26f9783 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:45:47.075500 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3aee7288-70b2-436d-8db8-30ded26f9783 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb8a8a4e8b634322a92992379c98b721'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:47.076240 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3aee7288-70b2-436d-8db8-30ded26f9783 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb8a8a4e8b634322a92992379c98b721'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:47.083147 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3aee7288-70b2-436d-8db8-30ded26f9783 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb8a8a4e8b634322a92992379c98b721'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:47.146541 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3aee7288-70b2-436d-8db8-30ded26f9783 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb8a8a4e8b634322a92992379c98b721'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:47.147175 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-3aee7288-70b2-436d-8db8-30ded26f9783 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb8a8a4e8b634322a92992379c98b721'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:47.169145 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1457/2897] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:45:46 2022] DELETE /identity/v3/users/bb8a8a4e8b634322a92992379c98b721 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:45:47.509632 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98991a00-afe8-42cb-afda-cacf8b65295e service neutron] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:45:47.566875 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98991a00-afe8-42cb-afda-cacf8b65295e service neutron] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:45:47.592171 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=100 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:45:47.624400 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51eda5-4f68-471c-a8ce-d7d60edfecf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:45:47.625267 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51eda5-4f68-471c-a8ce-d7d60edfecf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:45:47.625580 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51eda5-4f68-471c-a8ce-d7d60edfecf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:45:47.625857 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51eda5-4f68-471c-a8ce-d7d60edfecf8 admin admin] PATH_INFO: `/v3/projects/99847b649bfd4ca6a33cb2ea97ad0469` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:45:47.627051 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51eda5-4f68-471c-a8ce-d7d60edfecf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99847b649bfd4ca6a33cb2ea97ad0469)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:45:47.629008 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b51eda5-4f68-471c-a8ce-d7d60edfecf8 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:45:47.754902 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3b51eda5-4f68-471c-a8ce-d7d60edfecf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99847b649bfd4ca6a33cb2ea97ad0469', 'request_id': 'req-3b51eda5-4f68-471c-a8ce-d7d60edfecf8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:47.761803 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3b51eda5-4f68-471c-a8ce-d7d60edfecf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99847b649bfd4ca6a33cb2ea97ad0469', 'request_id': 'req-3b51eda5-4f68-471c-a8ce-d7d60edfecf8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:47.767662 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3b51eda5-4f68-471c-a8ce-d7d60edfecf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99847b649bfd4ca6a33cb2ea97ad0469', 'request_id': 'req-3b51eda5-4f68-471c-a8ce-d7d60edfecf8'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:47.856771 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3b51eda5-4f68-471c-a8ce-d7d60edfecf8 admin admin] The token cache is being invalidate because project 99847b649bfd4ca6a33cb2ea97ad0469 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:45:47.857048 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3b51eda5-4f68-471c-a8ce-d7d60edfecf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:47.857602 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-3b51eda5-4f68-471c-a8ce-d7d60edfecf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:47.859249 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1441/2898] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:45:47 2022] DELETE /identity/v3/projects/99847b649bfd4ca6a33cb2ea97ad0469 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:45:47.863249 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee7288-70b2-436d-8db8-30ded26f9783 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:45:47.864737 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-3aee7288-70b2-436d-8db8-30ded26f9783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:45:47.879062 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee7288-70b2-436d-8db8-30ded26f9783 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:45:47.939033 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241abbb0-5024-4c3a-a176-cb191f140e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:45:47.939933 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241abbb0-5024-4c3a-a176-cb191f140e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:45:47.940214 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241abbb0-5024-4c3a-a176-cb191f140e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:45:47.940493 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241abbb0-5024-4c3a-a176-cb191f140e88 admin admin] PATH_INFO: `/v3/users/f6aae1ef404f4511a8bd234cff0eac5b` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:45:47.941662 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241abbb0-5024-4c3a-a176-cb191f140e88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6aae1ef404f4511a8bd234cff0eac5b)` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:45:47.943614 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241abbb0-5024-4c3a-a176-cb191f140e88 admin admin] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:45:47.957388 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:45:48.166644 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-241abbb0-5024-4c3a-a176-cb191f140e88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6aae1ef404f4511a8bd234cff0eac5b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:48.168039 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-241abbb0-5024-4c3a-a176-cb191f140e88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6aae1ef404f4511a8bd234cff0eac5b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:48.175603 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-241abbb0-5024-4c3a-a176-cb191f140e88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6aae1ef404f4511a8bd234cff0eac5b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:48.261976 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-241abbb0-5024-4c3a-a176-cb191f140e88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6aae1ef404f4511a8bd234cff0eac5b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:48.262631 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.notifications [None req-241abbb0-5024-4c3a-a176-cb191f140e88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6aae1ef404f4511a8bd234cff0eac5b'} {{(pid=80298) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:48.281050 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1458/2899] 10.222.0.67 () {58 vars in 1223 bytes} [Thu Jun 30 16:45:47 2022] DELETE /identity/v3/users/f6aae1ef404f4511a8bd234cff0eac5b => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:45:48.513077 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51eda5-4f68-471c-a8ce-d7d60edfecf8 admin admin] Authenticating user token {{(pid=80297) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:45:48.518996 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.fernet_utils [None req-3b51eda5-4f68-471c-a8ce-d7d60edfecf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80297) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:45:48.528879 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b51eda5-4f68-471c-a8ce-d7d60edfecf8 admin admin] Validating token access rules against request {{(pid=80297) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:45:48.586517 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d8a09-cb77-4a49-9357-33bcf793a774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9242c0a59f6b49568f911aaa0ef688f2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3967c884c6b647c2a8e60d73c62e7927', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80297) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:45:48.587380 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d8a09-cb77-4a49-9357-33bcf793a774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:45:48.587695 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d8a09-cb77-4a49-9357-33bcf793a774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:45:48.587972 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d8a09-cb77-4a49-9357-33bcf793a774 admin admin] PATH_INFO: `/v3/projects/3d71b79e0b9841fc8392ca2fcb24b55f` {{(pid=80297) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:45:48.589650 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d8a09-cb77-4a49-9357-33bcf793a774 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d71b79e0b9841fc8392ca2fcb24b55f)` {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:45:48.591591 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d8a09-cb77-4a49-9357-33bcf793a774 admin admin] RBAC: Authorization granted {{(pid=80297) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:45:48.709439 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-526d8a09-cb77-4a49-9357-33bcf793a774 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d71b79e0b9841fc8392ca2fcb24b55f', 'request_id': 'req-526d8a09-cb77-4a49-9357-33bcf793a774'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:48.715921 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-526d8a09-cb77-4a49-9357-33bcf793a774 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d71b79e0b9841fc8392ca2fcb24b55f', 'request_id': 'req-526d8a09-cb77-4a49-9357-33bcf793a774'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:48.719895 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:45:48.721969 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-526d8a09-cb77-4a49-9357-33bcf793a774 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d71b79e0b9841fc8392ca2fcb24b55f', 'request_id': 'req-526d8a09-cb77-4a49-9357-33bcf793a774'} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:48.798714 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-526d8a09-cb77-4a49-9357-33bcf793a774 admin admin] The token cache is being invalidate because project 3d71b79e0b9841fc8392ca2fcb24b55f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80297) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 16:45:48.798945 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-526d8a09-cb77-4a49-9357-33bcf793a774 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:48.799570 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG keystone.notifications [None req-526d8a09-cb77-4a49-9357-33bcf793a774 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80297) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 16:45:48.801119 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: [pid: 80297|app: 0|req: 1442/2900] 10.222.0.67 () {58 vars in 1232 bytes} [Thu Jun 30 16:45:48 2022] DELETE /identity/v3/projects/3d71b79e0b9841fc8392ca2fcb24b55f => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 16:45:58.277511 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=22,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:45:58.959157 dv-cinder-836027-2-iscsi devstack@keystone.service[80297]: DEBUG dbcounter [-] [80297] Writing DB stats keystone:DELETE=2,keystone:SELECT=3,keystone:INSERT=1 {{(pid=80297) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 16:46:54.188109 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241abbb0-5024-4c3a-a176-cb191f140e88 admin admin] Authenticating user token {{(pid=80298) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 16:46:54.190541 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-241abbb0-5024-4c3a-a176-cb191f140e88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:46:54.304652 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241abbb0-5024-4c3a-a176-cb191f140e88 admin admin] Validating token access rules against request {{(pid=80298) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 16:46:54.369816 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd684eaf-a5b0-4d9c-8e72-d5e2906f90e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a932a9390fdd47f3adcf053a60250944', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '19ef92ebcce94b90bf39ad3a58772a1d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80298) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 16:46:54.370472 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd684eaf-a5b0-4d9c-8e72-d5e2906f90e5 service placement] REQUEST_METHOD: `GET` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 16:46:54.370762 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd684eaf-a5b0-4d9c-8e72-d5e2906f90e5 service placement] SCRIPT_NAME: `/identity` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 16:46:54.371044 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd684eaf-a5b0-4d9c-8e72-d5e2906f90e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80298) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 16:46:54.372959 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.fernet_utils [None req-fd684eaf-a5b0-4d9c-8e72-d5e2906f90e5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80298) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 16:46:54.385153 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd684eaf-a5b0-4d9c-8e72-d5e2906f90e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 16:46:54.386747 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd684eaf-a5b0-4d9c-8e72-d5e2906f90e5 service placement] RBAC: Authorization granted {{(pid=80298) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 16:46:54.397107 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: [pid: 80298|app: 0|req: 1459/2901] 10.222.0.67 () {60 vars in 1365 bytes} [Thu Jun 30 16:46:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 16:47:04.392645 dv-cinder-836027-2-iscsi devstack@keystone.service[80298]: DEBUG dbcounter [-] [80298] Writing DB stats keystone:SELECT=74 {{(pid=80298) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}